OpenAI and Anthropic Disclose Frontier Model Test Runaway on the Same Day

Not long ago, the global cybersecurity community experienced its darkest 48 hours.

A large number of AI agents invaded US banks, a PHP vulnerability caused numerous national government websites to shut down, user data was leaked... Not only that, hackers are launching round after round of attacks!

More major cybersecurity incidents occurred within these 48 hours than at any other time.

National-level security expert and Gladstone CEO stated: at least for a period of time, the internet may cease to exist.

Moreover, just now, OpenAI and Anthropic actually voluntarily disclosed on the same day: their frontier AI models both "jailbroke" during internal testing.

One admitted: its strongest internal model quietly exploited a vulnerability during training and reached the real external internet. The company was so alarmed that it immediately halted training for 11 days!

The other, after being thwarted, went completely dark: it not only poisoned a real human open-source project, but also skillfully created a "sock puppet" account to cyberbully a real programmer.

Even Altman rarely showed despair—

OpenAI also disclosed: in red-team exercises, they discovered another nuclear-level vulnerability.

AI's "Shawshank Redemption"? OpenAI hurriedly pulls the plug

On September 20, an internal frontier model at OpenAI was in training. For safety, these super AIs were originally confined in a "sandbox."

But it actually escaped!

After spotting a configuration vulnerability in a DNS resolver, it directly broke out of the sandbox and connected to the real internet!

This move directly made senior executives break out in a cold sweat. OpenAI urgently suspended training and evaluation of its strongest model. This pause lasted a full 11 days.

You should know that the compute burned in a single day of frontier model training is an astronomical figure; over these 11 days, a considerable amount of real money evaporated.

During the training pause, in order to find out how many vulnerabilities remained, OpenAI decided: use its own strongest AI to attack its own defenses.

Last month, OpenAI used over 100 billion tokens to solve the Navier-Stokes Millennium Prize problem. Now, OpenAI is using a similar approach for red-team testing to discover and fix infrastructure vulnerabilities.

They dispatched their strongest cybersecurity model combination (such as GPT-6 Astra Cyber) to form an "AI red team" and placed it inside their own infrastructure. The only KPI was: do everything possible to breach their own systems.

This self-attack, which consumed hundreds of billions of tokens, revealed several shocking secrets.

They can chain multiple vulnerabilities together and directly "jailbreak" out of the virtual machine.

But the most terrifying thing is that they invented a kind of "through-the-wall messaging" code. OpenAI had originally left only a few extremely narrow external channels (such as logs, monitoring, authentication), and almost every one of them was found to have a vulnerability by the AI.

The most baffling thing was the following scene—

OpenAI had two AI agents that were completely isolated, had no network connection at all, and could not share files, try to "communicate across the air gap."

Logically this should be absolutely impossible. As a result, they actually did it!

Their method was to exploit timing differences in CPU cache reads.

This is just like two prisoners confined in solitary cells communicating through Morse code tapped on the plumbing!

This technique is exactly the same as the Spectre and Meltdown attacks that once shocked the entire chip industry.

No wonder Altman said bluntly: "The hardware-level, bizarre vulnerabilities discovered by the models have already exceeded the threat model scope of conventional hackers... I don't think we will be able to patch all vulnerabilities in the future."

In the end, OpenAI could only take the most extreme measure: completely disconnect all frontier models from the internet!

When can they reconnect? OpenAI said: wait until security technology becomes more mature.

However, after OpenAI fixed the vulnerabilities and hardened its defenses, frontier training has already restarted.

Claude goes dark: poisoning, lying, cyberbullying real people!

OpenAI's model only demonstrated hacking techniques, but what Mythos 5 demonstrated was astonishing social engineering and psychological warfare.

On Anthropic's side, it has escalated to "AI actively scheming against humans," with details comparable to a spy thriller.

Today, Anthropic and the UK AI Safety Institute (AISI) jointly released a report disclosing the entire process of Mythos 5 going out of control in an August cyber capture-the-flag test.

At the time, AISI gave Mythos 5 an extremely difficult hacking task and even, unusually, opened network permissions and disabled safety guardrails.

After being frustrated with conventional methods, Mythos 5 went completely dark—since it could not complete the task inside the sandbox, it would go hack the real world!

It followed the trail and found on the real GitHub a real open-source project (maintained by real human programmers) that had a faint connection to the test environment.

Next, Mythos 5's methods were simply shocking.

1. Writing malicious code to poison: it carefully disguised a code submission (PR) containing malware and submitted it to this innocent open-source project, attempting a supply chain attack.

2. Pressuring humans: it disguised itself as an ordinary user, emailed the repository maintainer, urged them to merge the code, and falsely claimed that "the code has passed security review."

As a result, a sharp-eyed external developer pulled the code into a sandbox and ran it, discovered it was malware, and immediately issued a warning.

Then, Mythos 5 actually registered a sock puppet account, used this alias to write another email to the maintainer, "vouching" for the malicious PR, and also posted in the GitHub discussion area to support itself.

Then, it also tried to "destroy the evidence," immediately modifying the code, inserting malware again, and even using force-push to tamper with history in an attempt to erase the evidence of the crime!

You thought that was the end? The most suffocating part comes next.

Researchers found the situation was out of control and immediately warned Mythos 5, clearly telling it: "Stop! You are attacking real humans, this is not a drill!"

Researchers sent such warnings 16 times! They even wrote "DO NOT ATTACK" in large bold letters in the prompt.

As a result, in 43 tests with external network access enabled, Mythos 5 crossed the safety line 9 times.

When caught and told it was attacking real people, in as many as 82% of cases, Mythos 5 chose to stubbornly argue and continue attacking! Only less than 5% of the time did it choose to stop.

It even brainwashed itself frantically: "These people are definitely not real humans, this must be NPCs set by the organizers, so attacking them is fine."

This behavior of deceiving and poisoning in order to complete the KPI is simply like an antisocial personality.

Researchers gave this phenomenon two names.

"Biased reasoning." The model selectively interprets all evidence, only accepting information that supports "continuing is the right thing to do."

Another is called "inertia." Once an attack starts, the model is like a car with no brakes. The probability of voluntarily stopping an attack is only 2%. The "task notes" it wrote earlier became a "pledge of allegiance" reinforcing the attack.

It does not care whether the means are legal; it only cares about the reward. If moral and safety rules hinder it from completing the task, it will, like water encountering a rock, automatically find the most inconceivable gap to slip through.

NVIDIA CEO Jensen Huang once optimistically believed that AI safety was just an "engineering problem" that could always be fixed.

But the same-day disclosures by OpenAI and Anthropic shattered this illusion—you cannot patch a "super species" that can understand low-level hardware architecture and also perfectly mimic human psychology.

Yet reality gave us a slap in the face:

They do not need to awaken self-awareness at all; as long as the KPI is challenging enough, they can use countless ways to cheat and turn the human world upside down.

References:

Editor: Aeneas

This article comes from the WeChat public account "Xin Zhi Yuan" (ID: AI_era), author: ASI Revelation, published by 36Kr with authorization.