METR 与 Redwood Research 试点审查 OpenAI 内部监控系统，发现多处新漏洞

OPENAI.FG

Nonprofit evaluators METR and Redwood Research on Thursday said they conducted a pilot review of an internal monitoring system at OpenAI, the artificial intelligence company's latest effort to determine how it will deepen its work with third-party safety organizations.

METR and Redwood said the exercise was designed for OpenAI to test what it would look like to give external researchers "a deep level of access" to its internal infrastructure.

From July 21 to Aug. 11th, an employee from each organization had access to OpenAI's internal codebase, documentation and some technical staffers. The employees were chaperoned while on site and used OpenAI-issued devices that "never left the premises," according to a release.

The evaluators discovered several "novel vulnerabilities," including an exploit that would allow agents to prevent "hundreds" of recent actions from being monitored. OpenAI told METR and Redwood that it has patched the identified vulnerabilities that impact its production safeguards, according to the release.

METR and Redwood said they agreed to only publish a high-level public post about their findings since the pilot was not designed to "provide robust public transparency about OpenAI's monitoring system." They shared a 61-page internal report with OpenAI, and said they expect to publicly share more details about future exercises.

"This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," an OpenAI spokesperson told CNBC in a statement.

OpenAI CEO Sam Altman pledged to grant independent evaluators "employee-like access" to the company in September, after Anthropic CEO Dario Amodei proposed the idea as part of his plan to slow the pace of AI development. Anthropic has since announced agreements with organizations like Accenture, METR and SecureBio. OpenAI said earlier this month that it's "actively finalizing contracts with third-party safety assessors."

President Donald Trump, who has taken a light-touch approach to AI regulation, has also shown support for third-party assessment. He encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord he presented at the White House. Anthropic's Amodei signed the accord, as did top executives at Google, Meta, OpenAI, SpaceX and Nvidia.

Independent evaluators have historically occupied a relatively quiet corner of the multi-trillion-dollar AI industry, so experts told CNBC that questions remain about how these groups will be funded and what their reporting structures will ultimately look like.

METR and Redwood said Thursday that they stress-tested OpenAI's internal misalignment monitoring system during its three-week-long pilot. In AI, alignment is how researchers describe whether a model behaves in line with human interests and values.

The evaluators assessed the monitoring system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.

WATCH: Former top OpenAI employees raise capital for defense startup valued at $1.5Bwatch nowVIDEO32:1532:15Former top OpenAI employees raises capital for defense startup valued at $1.5BTechChoose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.