OpenAI 与 Apollo Research 等四家机构开展嵌入式评估
@ApolloResearch will assess whether our models could covertly work against their developers’ intentions—and whether our safeguards would detect it.
OpenAI 宣布与 Apollo Research、METR、TransluceAI 和 GoodfireAI 合作开展嵌入式评估,让外部专家深入接触其内部模型和系统,调查对齐问题并测试安全防护措施。其中 Apollo Research 将评估模型是否会暗中违背开发者意图行事,以及防护措施能否检测到这类行为,评估依据包括训练、安全测试和内部使用中的证据。METR 将调查其迄今最大规模强化学习训练中的失准行为,TransluceAI 将评估后训练对齐评测的质量与覆盖范围并计划在 2027 年扩大试点,GoodfireAI 则派驻可解释性研究人员分析模型内部。
推荐理由
AI 评分 58
OpenAI 公开四家外部机构各自负责的评估范围,读者可据此了解外部专家如何介入内部模型与系统。
主题
正文
@ApolloResearch will assess whether our models could covertly work against their developers’ intentions—and whether our safeguards would detect it.
It will examine evidence from training, safety testing, and internal use to test for this behavior and our ability to catch it.
AI 热点