Understand this story
Carnegie Mellon University published a paper proposing that pretraining, fine-tuning, and reinforcement learning (RL) can share the same training rule, and that mixing data from different stages often outperforms staged training. The paper reframes MaxRL as likelihood training on the model's own correct answers, enabling a single trainer to mix pretraining text, curated data, and RL samples in arbitrary proportions. On a 0.76B model, after mixing 5% midtraining data into the math RL stage, MBPP+ reached 32.69%, compared with 29.81% for staged training. The work is currently publicly available as a paper, with no subsequent progress yet.
AI-generated from public reports
Latest development · Report
CMU Paper: Pretraining, Fine-Tuning, and RL Can Be Unified into a Single Training RuleDevelopments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 31
Rohan Paul
CMU Paper: Pretraining, Fine-Tuning, and RL Can Be Unified into a Single Training Rule
A Carnegie Mellon University paper proposes that pretraining, fine-tuning, and RL can share the same training rule, and that mixing data from various stages often outperforms staged training. The authors reframe MaxRL as likelihood training on the model's own correct answers, enabling a single trainer to mix pretraining text, curated data, and RL samples in arbitrary proportions. On a 0.76B model, mixing 5% midtraining data into the math RL stage raised MBPP+ to 32.69%, compared with 29.81% for staged training.