Mira Murati：on-policy 蒸馏结合 RL 与 SFT 优势
Combining the benefits of RL and SFT with on-policy distillation, a promising approach for training small models for domain performance and continual learning.
AI 导读
结合 RL 和 SFT 的优势，通过 on-policy 蒸馏，这是一种有前景的方法，用于训练小模型以提升领域性能和持续学习能力。
查看原帖Our latest post explores on-policy distillation, a training approach that unites the error-correcting relevance of RL with the reward density of SFT. When training it for math reasoning and as an internal chat assistant, we find that on-policy distillation can outperform other approaches for a fraction of the cost. https://t.co/JhpyWQOpBe
正文
当前显示原文，中文全文译文尚未就绪。
Combining the benefits of RL and SFT with on-policy distillation, a promising approach for training small models for domain performance and continual learning.