Understand this story
On November 4, 2026, Ao Qu created the lecture slides for Session 6, "RL & Self-Distillation in Practice," of MIT's Fall 2026 course MAS.S62 "How to AI an AI." According to related reports, this lecture was taught by Idan Shenfeld of Thinking Machines Lab, and frames PPO, GRPO, on-policy distillation, and self-distillation as the same loss, differing only in the source of the advantage values; the slides break down the five stages of a single training step and, using Qwen3-8B as an example, give memory magnitudes of about 16 GB for bf16 weights on the inference side and about 16 bytes per parameter on the training side, totaling about 130 GB, and compare two frameworks: verl offline training and reef learning while serving. There are currently no further developments.
AI-generated from public reports
Latest development · Report
MIT "How to AI an AI" Lecture 6: From PPO and GRPO to On-Policy Distillation…Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 9
Shao Meng
MIT "How to AI an AI" Lecture 6: From PPO and GRPO to On-Policy Distillation…
The lecture slides for Lecture 6, "RL & Self-Distillation in Practice," of MIT's Fall 2026 course MAS.S62 "How to AI an AI" are taught by Idan Shenfeld of Thinking Machines Lab, and formulate PPO, GRPO, on-policy distillation, and self-distillation as a single loss, differing only in the source of the advantage values. The slides break down the five stages of a single training step, and using Qwen3-8B as an example, give the order of magnitude of memory: about 16 GB for bf16 weights on the inference side, and about 16 bytes per parameter on the training side, totaling about 130 GB; they also compare two frameworks: verl for offline training and reef for learning while serving.