AI News

OpenAI Conducts Embedded Alignment Evaluations with Four Institutions Including Apollo and METR

OpenAIUpdated Nov 17 00:13

AI summary

OpenAI is collaborating with Apollo Research, METR, TransluceAI, and GoodfireAI on embedded evaluations, giving external experts deep access to its internal models and systems to investigate alignment issues and test safety safeguards. Among them, METR will investigate misaligned behavior in OpenAI's largest reinforcement learning training to date, including failures that existing tests and monitoring may miss; Apollo Research will assess whether models secretly act against developer intent and whether safeguards can detect it; TransluceAI will review the quality and coverage of post-training alignment evaluations and plans to expand the pilot in 2027; GoodfireAI will embed interpretability researchers to analyze model internals. OpenAI also said it has preliminarily agreed to conduct broader monitoring evaluations, with the specific scope to be agreed upon by both parties.

Why read this

SelectedAI score 68

OpenAI has disclosed the division of alignment evaluation responsibilities among four external organizations, allowing readers to understand how external experts intervene in internal model reviews.

Topics

Tags

Body

@METR_Evals will investigate misaligned behavior in our largest reinforcement learning run to date, including failures our existing tests and monitors may have missed.

We’ve also preliminarily agreed to conduct a broader monitor assessment, pending mutual agreement on scope.

Source

OpenAI