Nathan Lambert comments on the reverse analysis of Tinker StableReinforce
AI summary
Martin Marek posted that Tinker's new StableReinforce objective is Reinforce plus Score Centering, and shared his own reverse-engineering process. Nathan Lambert reposted and commented that this is a good post, and that some RL details used by closed labs are slowly leaking out.
View original postTinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
View original
Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
Body
Nice thread, some RL details used by closed labs slowly leaking out.