AI News

Trending story

MIT course slides explain reinforcement learning and on-policy distillation practices

1 reports1 story updatesUpdated Nov 9 14:17

Understand this story

AI overview

On November 4, 2026, Ao Qu created the lecture slides for Session 6, "RL & Self-Distillation in Practice," of MIT's Fall 2026 course MAS.S62 "How to AI an AI." According to related reports, this lecture was taught by Idan Shenfeld of Thinking Machines Lab, and frames PPO, GRPO, on-policy distillation, and self-distillation as the same loss, differing only in the source of the advantage values; the slides break down the five stages of a single training step and, using Qwen3-8B as an example, give memory magnitudes of about 16 GB for bf16 weights on the inference side and about 16 bytes per parameter on the training side, totaling about 130 GB, and compare two frameworks: verl offline training and reef learning while serving. There are currently no further developments.

AI-generated from public reports

Latest development · Report

MIT "How to AI an AI" Lecture 6: From PPO and GRPO to On-Policy Distillation…

Developments

1 confirmed developments

  1. MIT course slides explain reinforcement learning and on-policy distillation in practice

    Source:Shao Meng

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 9

  1. Shao Meng

    MIT "How to AI an AI" Lecture 6: From PPO and GRPO to On-Policy Distillation…

    The lecture slides for Lecture 6, "RL & Self-Distillation in Practice," of MIT's Fall 2026 course MAS.S62 "How to AI an AI" are taught by Idan Shenfeld of Thinking Machines Lab, and formulate PPO, GRPO, on-policy distillation, and self-distillation as a single loss, differing only in the source of the advantage values. The slides break down the five stages of a single training step, and using Qwen3-8B as an example, give the order of magnitude of memory: about 16 GB for bf16 weights on the inference side, and about 16 bytes per parameter on the training side, totaling about 130 GB; they also compare two frameworks: verl for offline training and reef for learning while serving.

Related stories

  1. Related storiesIBM team integrates SGLang into Spyre using AI coding agent
  2. Related storiesEnglish translation is being prepared.
  3. Related storiesi14 migrates its custom architecture to AWS Trainium2
  4. Related storiesFireworks publishes Kimi K3 reinforcement learning infrastructure report