AI Trends

Nathan Lambert on the Public Evaluation Cycle

Nathan LambertUpdated Nov 19 01:39

AI summary

Responding to the claim that “public evaluations are all bad, while good evaluations are all private,” Nathan Lambert said he disagreed. He believes many public evaluations expand the boundaries of discussion, then become ineffective due to Goodhart’s law and are replaced by private evaluations; the key is which stage of the evaluation cycle they are in.

Quoted post

vik

@vikhyatk

if the eval is public, it's bad. all good evals are private

View original

if the eval is public, it's bad. all good evals are private

View original post

Body

I don’t agree with these statements. many public evals shift the Overton window to interesting new things. Then they are goodhart’d and private ones take over. Where you are in the cycle is important.

Source

Nathan Lambert