OpenAI Conducts Embedded Evaluations with Apollo Research and Three Other Institutions
OpenAI announced that it is collaborating with Apollo Research, METR, TransluceAI, and GoodfireAI to conduct embedded evaluations, giving external experts deep access to its internal models and systems to investigate alignment issues and test safety safeguards. Among them, Apollo Research will evaluate whether models might secretly act against developers' intentions, and whether safeguards can detect such behavior, based on evidence from training, safety testing, and internal use. METR will investigate misaligned behavior in its largest-scale reinforcement learning training to date, TransluceAI will evaluate the quality and coverage of post-training alignment evaluations and plans to expand the pilot in 2027, and GoodfireAI will embed interpretability researchers to analyze model internals.
Why read this
AI score 58
OpenAI has disclosed the scope of evaluation each of the four external institutions is responsible for, so readers can understand how external experts engage with internal models and systems.
Topics
Body
@ApolloResearch will assess whether our models could covertly work against their developers’ intentions—and whether our safeguards would detect it.
It will examine evidence from training, safety testing, and internal use to test for this behavior and our ability to catch it.
AI Trends