Nathan Lambert comments on the importance sampling table in the score-centering RL paper
AI summary
Nathan Lambert said he read the score-centering RL paper for his book notes and found the table in its appendix on importance-sampling-based methods for training-inference mismatch in RL to be excellent. He also believes that since the rise of GRPO, such methods have been the biggest focus of RL algorithm research.
Body
Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the different importance sampling based train-inference mismatch methods in RL. These have been the biggest focus of RL algorithm research since the GRPO takeoff, imo.