OpenAI 与 Apollo、METR 等四家机构开展嵌入式对齐评估
@METR_Evals will investigate misaligned behavior in our largest reinforcement learning run to date, including failures our existing tests and monitors may have
OpenAI 正与 Apollo Research、METR、TransluceAI 和 GoodfireAI 合作开展嵌入式评估,让外部专家深入接触其内部模型与系统,调查对齐问题并检验安全防护措施。其中 METR 将调查 OpenAI 迄今最大规模强化学习训练中的失准行为,包括现有测试和监控可能漏掉的失败;Apollo Research 评估模型是否会暗中违背开发者意图以及防护能否发现;TransluceAI 审查后训练对齐评估的质量与覆盖范围并计划在 2027 年扩大试点;GoodfireAI 则派驻可解释性研究人员分析模型内部。OpenAI 还表示已初步同意开展更广泛的监控评估,具体范围待双方商定。
推荐理由
精选AI 评分 68
OpenAI 公开了四家外部机构各自负责的对齐评估分工,读者可据此了解外部专家如何介入内部模型审查。
主题
正文
@METR_Evals will investigate misaligned behavior in our largest reinforcement learning run to date, including failures our existing tests and monitors may have missed.
We’ve also preliminarily agreed to conduct a broader monitor assessment, pending mutual agreement on scope.