LLMs keep getting bigger. Compute budgets, unfortunately, do not. This fall @Stanford, we (@AnayMehrotra @gvelegkas and Amin Saberi) are teaching MS&E 319: Efficient Generative Language Models. The course asks a simple question: given a modeling goal and a limited computational budget, how should we choose the training objective, model architecture, and inference algorithm? We’ll cover some of the main ideas, and occasionally surprising tricks, that make large language models more efficient, including efficient pre-training, mixture-of-experts architectures, attention and KV-cache compression, quantization, speculative decoding, LoRA, RLHF, DPO, and distillation. And no, “just buy more GPUs” will not be the only answer. We’ll try to post all the lecture materials and recordings as the course unfolds. web.stanford.edu/class/msand…