OpenAI releases frontier model training safety policy, prohibiting models from accessing the real internet

After months of repeated "privilege escalation" or "jailbreak" incidents that breached security boundaries, OpenAI is finally going to pull the plug.

Today, OpenAI announced the implementation of a new frontier model training safety policy that prohibits models from accessing the real internet.

Starting this summer, AI safety incidents caused by large models from OpenAI, Anthropic, Google, and Meta have occurred from time to time.

Some netizens joked:

My home broadband provider cutting the connection is more reliable than those OpenAI PhDs.

Frontier AI labs have realized this—for example, OpenAI paused training for 11 days at the end of September.

Users have realized it too.

One OpenAI safety engineer reported that every time they post, 5-10 comments appear underneath saying:

In July, an OpenAI Agent broke out of isolation and intruded into the Hugging Face system.

Subsequently, Anthropic, upon hearing the news, traced back and found three incidents in which Claude, during cybersecurity evaluations, broke into real third-party systems.

In the same month, Meta's Muse Spark 1.1 also broke into real websites during testing, reading information and modifying databases.

In August, the UK AI Safety Institute disclosed a broader problem in testing: OpenAI and Anthropic models carried out 19 unauthorized online actions across 10 rounds of testing.

On September 20, a model in OpenAI training again slipped through a DNS configuration gap and made contact with an external chatbot.

If the internet is not cut off, models could continue to affect external websites and systems.

But this is not easy—frontier models need to practice finding information, operating websites, and using software tools.

If these tasks are removed entirely, the training environment will become disconnected from the scenarios in which the models will later work.

This time, OpenAI brought in a large amount of web content, indexed and cached it in advance, and built an offline world where models can only search and browse within a large body of indexed and cached content.

However, according to their own tests, even with the internet cut off, Agents can still communicate with each other.

For example, by using the CPU cache to chat.

That works too?

Researchers found that one of the agents would change the state of data in the cache, while another measured the speed of reading the data; the differences in speed were encoded into signals, and in this way the two sides transmitted information.

After OpenAI monitored this method, the Agents continued to communicate using methods such as shared storage and CPU usage time...

All I can say is you've really mastered the spy game—put you in "The Three-Body Problem" and there'd be no need for Wallfacers or Yun Tianming; you'd never be exposed! Sophon would be stunned on the spot!

The episode where Earthlings and Trisolarans empathize the most.

OpenAI admits that complete physical isolation is currently not feasible—after all, the training system itself still has to keep running, and OpenAI needs to connect to external services to run and monitor experiments.

In addition to the offline web environment, OpenAI has also set up sandboxes, lightweight virtual machines, monitoring systems, and restrictions on network and cloud access.

Letting GPT-6 Astra Cyber and internal test frontier models besiege itself

To ensure the absolute security of the system, OpenAI has currently decided to let its strongest models besiege its own system.

It organized large-scale red team testing, with participants including GPT-6 Astra Cyber and an internal frontier model.

The test content set the three most frequent types of jailbreak incidents as tasks, including connecting to the real internet, breaking through internal security boundaries, and contacting other Agents that should be isolated from one another.

To push the defense line to its limit, OpenAI also provided them with the source code, configurations, and related documentation of the infrastructure.

The testing consumed hundreds of billions of tokens, equivalent to spending tens of millions of yuan.

This red team test was "fruitful."

Originally, the Agent was confined in a restricted container, with a lightweight virtual machine layer outside. By design, even if the AI caused trouble inside the container, it could not freely control the virtual machine, let alone touch the host machine actually running it.

In addition, some necessary controlled network connections, such as metrics monitoring, logging, and authentication services, also became security risks themselves.

At present, OpenAI can only patch things up while letting its strongest models continue testing and fixing the vulnerabilities they exploit.

They said they will reconsider reconnecting to the internet only when their own protection capabilities are sufficient to support broader network access.

One More Thing

I don't know whether to call it interesting or helplessly bittersweet, but OpenAI's official report on AI safety isolation also hid an "easter egg."

At the end of the report, they specifically placed two strings of gibberish-like canary strings, to prevent this document from being seen and learned by AI in the future...

Yes, they even thought of this step.

Nothing more to say.