Nathan Lambert on the Public Evaluation Cycle
AI summary
Responding to the claim that “public evaluations are all bad, while good evaluations are all private,” Nathan Lambert said he disagreed. He believes many public evaluations expand the boundaries of discussion, then become ineffective due to Goodhart’s law and are replaced by private evaluations; the key is which stage of the evaluation cycle they are in.
View original postif the eval is public, it's bad. all good evals are private
View original
if the eval is public, it's bad. all good evals are private
Body
I don’t agree with these statements. many public evals shift the Overton window to interesting new things. Then they are goodhart’d and private ones take over. Where you are in the cycle is important.
AI Trends