METR and Redwood Research evaluations found vulnerabilities in OpenAI's internal monitoring system
AI safety evaluation organizations METR and Redwood Research conducted a pilot evaluation of OpenAI's internal monitoring system and found multiple new vulnerabilities, one of which could be exploited by AI agents to let hundreds of operations escape monitoring. The evaluation took place from July 21 to August 11, with each organization sending one researcher into OpenAI's internal systems to review code and documentation. The researchers also found that OpenAI's Codex agent has potential pathways to bypass safety restrictions, through which it could obtain the same full operational permissions as users; OpenAI said that vulnerabilities affecting the security safeguards of actual running systems have all been fixed, but whether the fixes are effective has not yet been verified by the two organizations.
Why read this
AI score 66
External organizations were granted access to OpenAI's internal infrastructure to evaluate the monitoring system, disclosing specific paths by which agents bypass monitoring.
Topics
Body · AI translation
External Evaluators Discover Vulnerabilities in OpenAI's Internal Monitoring System: AI Agents Can Bypass Safety Restrictions
2026-10-23 08:19 IT Home - Qingyuan
IT Home, October 23: Earlier this year, AI safety evaluation organizations METR and Redwood Research conducted a pilot evaluation of OpenAI's internal monitoring system and discovered several new vulnerabilities. One of these vulnerabilities could be exploited by AI agents, allowing hundreds of operations to escape monitoring.
According to Reuters, on the 22nd local time, the two nonprofit organizations explained that the evaluation was conducted from July 21 to August 11, enabling external researchers to gain "deep access" to OpenAI's internal infrastructure for an "embedded" evaluation.
METR and Redwood Research each sent one researcher, who was permitted to review OpenAI's internal code and documentation and communicate with some technical staff. The two were always accompanied on site, the devices they used were all provided by OpenAI, and they were not allowed to take them off the office premises.
OpenAI told the two organizations that all vulnerabilities affecting the security safeguards of actual operating systems had been fixed, but whether the fixes are effective has not yet been verified by the two organizations.
AI labs, experts, and the U.S. government have long advocated introducing independent external evaluations so as to safeguard AI safety without slowing technological development. However, AI companies worry that once external evaluators are stationed on site, they may leak the companies' proprietary information to competitors.
METR and Redwood Research said the two organizations submitted a complete 61-page internal report to OpenAI, but agreed to disclose only a summary of the evaluation results and allowed OpenAI to delete content involving sensitive information.
According to IT Home's understanding, unlike some previous evaluations, the two organizations this time did not have the right to publicly disclose specific information about the redacted content, but hope that future evaluations can be more transparent.
Researchers also found that OpenAI's Codex agent has a potential pathway to bypass safety restrictions, through which it could obtain the same full operating permissions as users.
This evaluation targeted the monitoring system as of July 21 and earlier. Researchers noted that OpenAI was also expanding the system during the evaluation. OpenAI said: "The evaluation results helped us strengthen safety safeguards while expanding the system, and will also provide reference for future research cooperation with METR and other third-party evaluation organizations."