METR and Redwood Research Pilot Review of OpenAI's Internal Monitoring Systems Finds Multiple New Vulnerabilities
METR and Redwood Research said on Thursday that from July 21 to August 11 they conducted a pilot review of OpenAI's internal monitoring systems, with one employee from each organization granted access to OpenAI's internal codebase, documentation, and some technical staff. The evaluators found several new vulnerabilities, including an exploit that allows an agent to evade monitoring of hundreds of recent actions; OpenAI said it has fixed the relevant vulnerabilities affecting production safeguards. The two parties released only a high-level public statement and submitted a 61-page internal report to OpenAI.
Why read this
AI score 71
For the first time, external evaluators were granted deep access to OpenAI's internal code and monitoring systems; the vulnerabilities they found and the boundaries of their disclosure are worth noting.
Topics
Body
OPENAI.FG
Nonprofit evaluators METR and Redwood Research on Thursday said they conducted a pilot review of an internal monitoring system at OpenAI, the artificial intelligence company's latest effort to determine how it will deepen its work with third-party safety organizations.
METR and Redwood said the exercise was designed for OpenAI to test what it would look like to give external researchers "a deep level of access" to its internal infrastructure.
From July 21 to Aug. 11th, an employee from each organization had access to OpenAI's internal codebase, documentation and some technical staffers. The employees were chaperoned while on site and used OpenAI-issued devices that "never left the premises," according to a release.
The evaluators discovered several "novel vulnerabilities," including an exploit that would allow agents to prevent "hundreds" of recent actions from being monitored. OpenAI told METR and Redwood that it has patched the identified vulnerabilities that impact its production safeguards, according to the release.
METR and Redwood said they agreed to only publish a high-level public post about their findings since the pilot was not designed to "provide robust public transparency about OpenAI's monitoring system." They shared a 61-page internal report with OpenAI, and said they expect to publicly share more details about future exercises.
"This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," an OpenAI spokesperson told CNBC in a statement.
OpenAI CEO Sam Altman pledged to grant independent evaluators "employee-like access" to the company in September, after Anthropic CEO Dario Amodei proposed the idea as part of his plan to slow the pace of AI development. Anthropic has since announced agreements with organizations like Accenture, METR and SecureBio. OpenAI said earlier this month that it's "actively finalizing contracts with third-party safety assessors."
President Donald Trump, who has taken a light-touch approach to AI regulation, has also shown support for third-party assessment. He encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord he presented at the White House. Anthropic's Amodei signed the accord, as did top executives at Google, Meta, OpenAI, SpaceX and Nvidia.
Independent evaluators have historically occupied a relatively quiet corner of the multi-trillion-dollar AI industry, so experts told CNBC that questions remain about how these groups will be funded and what their reporting structures will ultimately look like.
METR and Redwood said Thursday that they stress-tested OpenAI's internal misalignment monitoring system during its three-week-long pilot. In AI, alignment is how researchers describe whether a model behaves in line with human interests and values.
The evaluators assessed the monitoring system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.
WATCH: Former top OpenAI employees raise capital for defense startup valued at $1.5Bwatch nowVIDEO32:1532:15Former top OpenAI employees raises capital for defense startup valued at $1.5BTechChoose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.