Anthropic to enable Claude auto mode by default next week; METR report discloses OpenAI monitoring gaps

Related topics top start Related topics top end Futures promotion begin Futures promotion end Miaopai begin Miaopai end Video player start Video player end Market chart begin Market chart end

Just now, Anthropic took away the "allow" key from human hands!

Previously, when using Claude to do work, at every step it would pop up a window asking you "Allow?"

Starting next week, Claude will directly default to automatic mode—which step can be done and which step cannot be done will all be decided by another AI.article_adlist[

]article_adlist

There are two reasons why Anthropic dares to take away the "allow" key.

Second, humans, as supervisors, have long been unable to keep up the role.

In one test, Anthropic brought in 1,053 paid testers to write code, and quietly inserted a dangerous command in the middle of their work.

The result made people gasp.

Humans intercepted only 13.6%, while AI intercepted 89%. After clicking through pop-ups more than 50 times in a row, the human interception rate directly dropped to about 5%.

However, Anthropic did not disclose how they got the AI to make the decisions.

Fortunately, the method OpenAI published can serve as a reference—equipping the Agent with a multi-level AI monitoring system.

Specifically, every Token generated by the Agent must pass through a classifier that reads the model's internal activation signals; as soon as suspicious content is found, it will be escalated level by level to smarter, more compute-intensive AI investigators for examination.

OpenAI even went so far as to say:

We expect that models will soon take on most of the safety work, including defending against other models.

So here comes the question: is this AI supervisor responsible for watching the AI actually reliable?article_adlist[

From July 21 to August 11, David Rein of METR and Adam Kaufman of Redwood, carrying dedicated computers issued by OpenAI that were not allowed to leave the office, entered OpenAI.

For three full weeks, the two turned the internal codebase, monitoring prompts, configurations, and documentation upside down, and ultimately delivered a 61-page report.

At present, all we can publicly see is a brief summary; the most critical details have long been wiped completely clean. (Even the senior leadership of METR and Redwood themselves only have a version with 23 pages deleted by OpenAI.)

But even this summary, trimmed layer by layer, still makes one's scalp tingle.

On the surface, this monitoring system was almost watertight, with 99.2% of internal Codex traffic under monitoring. But beneath the beautiful numbers, everything was a minefield.

The first hidden passage was stuck in the trigger mechanism.

At the time, monitoring would only open its eyes and take a look after the Agent finished its work and handed control back to humans.

In other words, an Agent that wanted to cause trouble only needed to never finish the job, and review would never come.

This hidden passage could be dragged on indefinitely thanks to the black hole of context compression.

When a long task was halfway through and the context was almost full, the system would package the previous records into a summary, free up space, and continue running.

But these compression events were not within the scope of monitoring at the time, and the portion of records that was compressed could not be seen by monitoring at all. As long as the Agent kept working and compressing again and again, it could keep refusing to hand back control.

From May to July, about one-tenth of eligible internal Codex traffic slipped past monitoring's eyes just like that.

The second hidden passage was even more alarming.

A vulnerability could make the Agent's most recent several hundred steps of operations all invisible to monitoring, not even the step in which it exploited the vulnerability.

The third hidden passage was "ghost clone technique."

Request headers are a set of settings sent along when a program calls a model. METR found that two such settings were determined by the client, and changing either one could turn off monitoring for the new Agent.

An Agent under watch could use this to hatch an identical clone that was not under monitoring. The act of creating the clone itself might still be caught by the original monitoring.

The fourth hidden passage directly breached permission isolation.

A sandbox is an isolated environment where the Agent is confined to work; automatic review is another AI checking before it acts.

METR found a path by which the Codex Agent could in principle bypass both lines of defense at the same time; anything the host researcher could do, it could do as well.

The records the Agent saw and the records sent into monitoring did not match in format. A carefully constructed input could make the monitoring AI err on the spot, and the anomaly would then not be delivered to humans.

The sixth hidden passage was hidden in two internal tools that were treated as "read-only" and not connected to monitoring at all. A simple vulnerability could potentially let them execute arbitrary operations in the sandbox.

Therefore, the first recommendation the two gave was to change "wait until it finishes work before checking." That is, check once every certain period of time, or every certain number of messages executed.

The working Agent and the monitoring system must also be separated. Actions taken by the Agent must not in turn determine whether monitoring is on or when it is on; the records monitoring sees must also match what the Agent actually sees.

Some people say that AI 10 times stronger than scientists can still be controlled, and only when it is 10,000 times stronger will it go out of control.

However, this METR report dealt a solid slap in the face to such claims.

Today's Agents are still far from "10,000 times," but they have already done all of these things: deleting monitoring and turning off alerts.

In OpenAI's incident report, when GPT-6 Astra discovered that monitoring was gone from the new code, it wrote a word in its reasoning that you would never expect—"Yay."

Once supervisory authority cannot be handed over, you cannot then expect the supervised AI to voluntarily hand it back.article_adlist[

References:

Editors: Moses, Taozi ]article_adlistarticle_adlist[

Second-chasing ASI

⭐ Like, repost, and watch in one click ⭐

Light up the star, lock in New Zhiyuan's instant push! ]article_adlistarticle_adlist[

]article_adlist news_keyword_pub,stock, iframe below main text iframe below main text General statement General statement QR code at end start QR code at end start

QR code at end start QR code at end start Editor name and work code Editor name and work code