Understand this story
On October 18, 2026, Shao Meng published an article introducing Lecture 10, "Inference," of Stanford University's CS336 course. According to the article, this lecture was taught by Percy Liang and uses a single mathematical thread to connect inference optimization at three levels—architecture, algorithms, and systems: at the architecture level, GQA/MLA/CLA reduce dimensionality, local attention truncation, quantization, and pruning and distillation compress the KV cache and parameters; at the algorithm level, speculative sampling achieves lossless acceleration; at the systems level, continuous batching, paging, prefix sharing, and copy-on-write are introduced. The article also notes that the lecture notes are in executable form, with formulas derived step by step by sympy and verified by assertions; Percy Liang believes that the KV cache is the structural root of the Transformer's unfriendliness to inference, and that architectures natively designed for inference (SSMs, linear attention, diffusion-based non-autoregressive generation) may offer the greatest room for improvement. Currently, this content has only one publication record, from Shao Meng.
AI-generated from public reports
Latest development · Report
Stanford CS336 Lecture 10: Percy Liang on Inference OptimizationDevelopments
1 confirmed developments
Shao Meng introduces Lecture 10 of Stanford CS336, Inference
Source:Shao Meng
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 18
Shao Meng
Stanford CS336 Lecture 10: Percy Liang on Inference Optimization
Stanford CS336 Lecture 10 "Inference" is taught by Percy Liang, who uses a single mathematical thread to connect three layers of inference optimization—architecture, algorithms, and systems: at the architecture layer, GQA/MLA/CLA reduce dimensionality, local attention truncates context, and quantization, pruning, and distillation compress the KV cache and parameters; at the algorithm layer, speculative sampling achieves lossless acceleration; at the systems layer, continuous batching, paging, prefix sharing, and copy-on-write are introduced. The lecture notes are in executable form, with formulas derived step by step by sympy and verified with assertions. Percy believes that the KV cache is the structural root of the Transformer's unfriendliness to inference, and that architectures natively designed for inference (SSMs, linear attention, diffusion-style non-autoregressive generation) may offer the greatest room for improvement.