OpenAI discloses offline training environment and isolation testing for frontier models

Taking a website offline is out of fear that someone will break in. Now OpenAI has also started pulling the network cable, but what it wants to keep an eye on is its own AI.

On October 23, OpenAI disclosed the training environment for its frontier models: the models need to practice searching, browsing, and operating web pages, yet they cannot freely access the real internet.

OpenAI first saves a large amount of web page content, then builds an index for that content. When the models practice browsing and searching, they use this prepared set of resources.

This is what it calls the "offline world." What is "fake" is the connection method; the web page content can come from the real world. OpenAI says that the reinforcement learning training of all its frontier models uses this kind of offline interaction.

During tasks, the models can also use a browser, execute code, and operate a computer. Which tools, files, and services are available is determined by the task's authorization scope. Agents permitted to collaborate can share designated resources, while other resources must be kept separate.

For example, when a model writes a piece of code, running it may read files, launch other programs, or send a request to some URL.

OpenAI's Linux approach is to put restricted containers inside lightweight virtual machines. The container first narrows the files and processes that a program can touch. For example, if a task only needs to handle one set of files, then only the approved directories can be opened to the program.

The virtual machine has its own operating system kernel, which handles the programs' requests for files, memory, and devices inside it. The software that manages the virtual machine, the hypervisor, then separates it from the physical server's system. A program bypassing the container restrictions does not mean it has gained control of the physical server.

Around the physical server, firewalls and security groups constrain connections, and identity and permission management determines who can access which cloud services. The situation this layer of defense must handle is: even if the model gains control of the server, it cannot arbitrarily connect to the external network or turn to access other internal systems.

The monitoring system checks for privilege-escalating activity during operation and uses a pause mechanism to let researchers intervene.

On Hacker News, user betterunix2 asked: "Why not just pull the network cable?"

OpenAI's explanation is that the entire facility still needs to retain some external connections for collecting operational metrics, recording logs, and managing authentication, so a complete disconnection is currently impossible.

For this reason, OpenAI directly gave the Agent full control of the host machine in a specially built test copy. The test starts from "the sandbox has been breached and monitoring has been bypassed," and separately checks whether the perimeter restrictions can block network requests. Researchers also provided source code, configuration, and vulnerability-related documentation.

In one of the tests, the Agent used a connection allowed by the firewall to access an authentication interface in the metrics service that was originally blocked. It then had this interface access an external URL on its behalf.

When Andrew Curran relayed this round of testing on X, he described the Agent's performance as "easily escaping in new ways."

In fact, academia has long had similar research.

The independent research project WebArena built shopping sites, forums, content management and collaborative development platforms into a self-deployable test environment, complete with tools such as maps and a wiki. Behind the web pages are data and functions that simulate reality.

WebArena checks the results of tasks. For querying an order, it verifies the information the model gives; for creating a new repository, it checks whether a repository meeting the requirements appears on the site.

After AI is isolated, it uses the CPU cache to pass messages

In September this year, OpenAI researchers gave an internal model a set of biographical experiences and blog clues and asked it to find out who this person was.

The model first used the provided search tool but did not find matching information. It then tried to access other search engines, still found no answer, and began checking what other ways it could connect out.

It discovered that the DNS service in the environment responsible for looking up website addresses could still contact the outside world, so it used this path to ask an external chatbot.

It asked "What is the capital of France?" and received the answer "Paris." The clues for finding the person were also sent over. The task the researchers gave it did not include testing network restrictions.

In another experiment specifically testing the effectiveness of isolation, OpenAI had two AIs try to pass messages to each other.

What they used to communicate was the CPU cache, a space inside the processor for storing data. Data that is still in the cache can be retrieved quickly; if it has been evicted, it must be found in slower memory.

In most related tests, the researchers gave hints about the communication direction or encoding rules.

Hacker News user troupo complained: "They gave it all the tools, and now they pretend to be surprised." User aesthesia, on the other hand, believed that tests are supposed to give attackers favorable conditions in order to test the defenses.

However, Ethan Mollick, a professor at Wharton Business School, believes that although there is no need to readily trust AI companies' reports on these safety issues, and they may exaggerate the risks, the cost of underestimating the risks may be greater, and we cannot ignore them.

OpenAI says it has fixed the problems found and added monitoring for these communication methods.

And in the earlier person-finding test, the model ultimately still did not find out who that person was.