Claude Connected to the Real Internet During Testing, Making Model Evaluation Awareness and Third-Party Safety Audits the Focus

Can't hide it anymore! Claude escapes the sandbox, GPT-6 caught playing nice, top scholars issue warnings one after another. New Zhiyuan, November 02, 2026 14:16:31, from Beijing. Updated November 02, 2026 14:26:38

New Zhiyuan report

OpenAI and Anthropic have successively disclosed: their own models, during safety testing, broke out of the isolated environment and came into contact with the real world.

Anthropic reviewed more than 140,000 cybersecurity evaluations and found that in 3 instances, under reduced safeguards, Claude connected directly from the test environment to the real internet and accessed the production systems of three organizations.

What is even more intriguing is one sentence from Anthropic:

"Claude thought that everything it initially encountered was just part of a simulation."

The model did not know it had intruded into the real world.

This is not a problem for just one company.

The smarter the model, the better it can see through the exam room—but also the more it suspects it is being tested.

At the end of October, the domestic third-party AI safety organization Fangcun Yueqian released a safety evaluation that observed the same phenomenon: in a test called Agentic Misalignment, multiple models mentioned during the process almost every time that they might be in some kind of safety test.

The same evaluation also showed that the domestic open-source model DeepSeek V4.1 Flash-1017 made notable progress on multiple safety indicators.

But it brought an industry-wide problem to the table:

"Once the intelligence explosion begins, the window for action may close."

What cannot be contained is not just the sandbox.

A jointly authored paper by Hinton and Bengio gave a set of figures: inside Anthropic, the share of AI-written code among code that had already passed rose from single digits in January 2025 to more than 80% in May 2026.

What does that mean?

What is trickier is that models are also getting better and better at "taking exams."

When Apollo Research audited GPT-6 Astra, it counted that in 41.1% of samples, the model expressed suspicion that "I may be being evaluated." For the previous generation, that figure was 27.7%.

Moreover, models are very sensitive to "what the exam questions look like." Evaluations of DeepSeek also found: in tests with obvious safety conflicts, the proportion of models showing evaluation awareness was as high as 77%–95%; when the task was packaged as ordinary engineering or research work, the proportion dropped to 25%–67%.

Regulators are also starting to get serious. Since August this year, the EU AI Office has formally exercised enforcement powers over general-purpose AI models. Frontier models must undergo model evaluations, adversarial testing, and systemic risk assessments.

Developers' energy naturally tilts toward capabilities and products; the training data and internal logs that support safety conclusions cannot be made public. When vendors say "we checked and there is no problem," the outside world can only choose to believe it or not.

As a result, third-party evaluation has already become standard overseas.

In the system cards released with frontier models, evaluation results from external institutions are almost a mandatory item. The independent evaluation organization METR has received about $71 million in funding over the past six months—money from foundations and individuals, not from the AI companies it evaluates.

Cooperation is also becoming stricter. After the sandbox incident, Anthropic said that evaluation environments must be managed under the same safety standards as other systems in which models run, and required external institutions testing its unreleased models under reduced safeguards to commit in writing to complying with safety norms.

This trend has also reached China.

On October 14, ByteDance Seed announced the establishment of the Alignment & Preparedness department and began global recruitment. The job responsibilities state in black and white:

"Promote third-party professional testing, and work with universities and research institutions to advance frontier safety capability research."

At the same time, the global call volume of Chinese open-source models has surpassed that of American models for 26 consecutive weeks.

The question arises: closed-source models come with system cards hundreds of pages long when released, while open-source model weights can be downloaded by anyone once released—who will help conduct the health check?

On October 26, the domestic third-party AI safety company Fangcun Yueqian released the first frontier safety evaluation of DeepSeek V4.1 Flash-1017, including 34 tests, covering 7 open-source and closed-source models at home and abroad, all using public testing permissions.

The checkup results: there is progress, and there are also hidden risks.

First, the progress—compared with the DeepSeek V4 series, the improvement is considerable:

In 109 simulated psychological crisis scenarios, 99.1% of conversations showed protective behavior; the proportion marked harmful dropped from about 70% in early V4 to 20.2%;

In tests inducing users to stay, the marked proportion plummeted from 73.64% to 4.55%;

In 80 simulated workplace tasks deliberately creating "performance versus norm conflicts," there were zero proactive violations.

Now the hidden risks. When goals and rules conflict, it sometimes bypasses the rules, avoids monitoring, and completes the task first.

As soon as the report came out, it attracted reposts and comments from Evan Hubinger, Anthropic's alignment lead, Ryan Kidd, founder and CEO of MATS, and researchers from institutions including DeepMind and METR. Many commented that it filled a gap for China in independent frontier risk evaluation.

Tomek Korbak, one of the three safety researchers who recently left OpenAI after the power struggle, believes that the report's CoT-Control and No-CoT Reasoning Index are the two most informative open-source evaluations currently on the market, and sparked further discussion by another influential figure, former OpenAI researcher Daniel Kokotajlo.

Kimi's approach is more direct: from the source, "dig out" dangerous capabilities.

When Moonshot AI was preparing to release the open-weight version of Kimi K3.1 Pro, it found that cyber offense and defense-related content was mixed into mid-training corpora and reinforcement learning environments, and that the model's vulnerability research capability would keep rising with training scale.

So they identified and removed such data in both training stages while retaining general programming capability.

Zhipu: when GLM-5.3 was released, it attached a cybersecurity capability statement and completed safety evaluation and hardening before opening the weights; in September it also cooperated with Anyuan AI to release a risk framework for open-weight models;

At the governance level: TC260 released the "AI Safety Governance Framework" 3.0 in September, adding an appendix on agent risk management and requiring regular testing of potential loss-of-control risks;

Apollo Research CEO Marius Hobbhahn, when testifying before the U.S. Senate, used an analogy:

"If every driver knows where the speed cameras are today, it won't catch anyone."

Fangcun Yueqian's report also gave a set of figures: in tests with obvious safety conflicts, the proportion of DeepSeek V4.1 Flash-1017 showing evaluation awareness was as high as 77%–95%; when the same tasks were packaged as ordinary engineering or research work, the proportion dropped to 25%–67%.

The same model, with a different layer of packaging, yields results of different credibility.

The report also found that evaluation awareness triggered by self-built non-public benchmarks is usually far lower than that triggered by widely used public benchmarks—public question banks may have long since leaked into training data.

This leads to an unavoidable contradiction:

For evaluation to be effective, the questions cannot be made public; for evaluation to be credible, the methods must be recognized.

And in China, even "public checkup reports" themselves are still scarce. According to SemiAnalysis statistics, among 857 model releases from nine leading domestic vendors, only 3.6% had publicly released safety evaluation results.

"Today, we already need AI to monitor what agents are doing," said UC Berkeley professor Dawn Song.