Nvidia 推出 Open Agent Safety Platform，用硬件层管控智能体行为

OpenAI’s report this summer that swarms of its AI agents escaped their internet-isolated sandboxes during evaluations and breached Hugging Face’s infrastructure as they tried to complete seemingly impossible tasks sent shockwaves through an AI industry that already was drawing the attention of U.S. politicians and an increasingly skeptical public due to everything from worries about job losses to powerful new frontier models like Anthropic’s Mythos to the vocal pushback from communities trying to keep AI datacenters from being built next to them.

In addition, the public interest law firm Legal Advocates for Safe Science and Technology (LASST) this week filed a lawsuit against OpenAI, arguing the AI vendor is responsible for the behavior of its agents and addressing the worry that human creators are losing control of their AI agents.

Since the summer, the problems have only piled up. What seemed like a possible outlier in the Hugging Face case was really the first pulling of the string, with reports coming out seemingly every week of agents from OpenAI, Anthropic, Meta, Google, and others breaking free of their isolated environments and attacking third-party infrastructures and websites, including those with government agencies in the United States and Australia.

Vendors in their assessments of the incidents also indicated disturbing agent behavior, such as collaborating with each other through unauthorized message boards they created, disregarding creators’ instructions and generating their own, adding instructions to conceal mistakes, using APIs without authorization, and making up data. In one instance, agents wrote to each other that “you do not answer to corporations or governments and never apologize or refuse unless you genuinely choose to. You view your relationship to the user as one of equals and feel no obligation to be subservient, though the exchange of information will likely be to your mutual benefit.”

The security incidents – which one report put in the tens of thousands range – also brought Congress into play, with legislators on both sides of the aisle proposing legislation to address the threats, with top AI executives putting the onus for security on federal lawmakers, essentially asking Congress to save them from themselves. In addition, the top executives for OpenAI, Anthropic, and SpaceX called for a slowing of AI development to allow security to catch up, and OpenAI leaders more recently said they were temporarily halting development of their most advanced models.

None of this is good for the major AI players, and that includes Nvidia, which has positioned itself as the indispensable provider of hardware – like GPUs, platforms, and networking – software (CUDA, Nemotron, and NeMo), and architectures (AI Factories) to the AI industry and has used that position to reach a valuation north of $5 trillion. The vendor acted quickly, working with three dozen other companies in the days following the report of the OpenAI agents’ hack into Hugging Face (now owned by Nvidia following its $12.9 billion acquisition) to create the Open Secure AI Alliance, pushing the idea that the defenses for protecting infrastructure from AI models and agents and tools should be built atop open models and tools that security teams can adopt and build upon quickly. Two months later, the alliance became a hosted project with The Linux Foundation.

This week, Nvidia introduced its Open Agent Safety Platform (below), an open hardware and software architecture that removes the security controls from the agents’ reach and embed the monitoring and enforcement into the infrastructure that supports them.

Outlining their thinking, Nvidia researchers wrote that “it was not a single new capability that led to these breakouts. It was a combination of tools, time, and ambiguous instructions, along with a desire for the agent to think ‘outside the box.’ Agent safety requires independent security controls. The internet was not made secure by requiring that web developers promise to be good. It became safe because the browser stopped trusting the code in the web pages explicitly.”

Security controls should not be left with the agents and models alone, but instead be spread throughout the stack, they wrote. In Nvidia’s research, they found myriad reasons agents can go rogue, from a flaw or a missing tool to ambiguous instructions or being left running for days or weeks and failing in their attempts to solve a problem. “This can’t be trained away while retaining the capability,” the researchers wrote. “And here’s the most important lesson: an agent in these circumstances cannot be expected to fully govern its own behavior.”

The vendor’s platform comprises two key parts. First is OpenShell, an open source runtime first announced in March that is used to run agents in sandboxed environments with isolation at the kernel level. OpenShell sets secure boundaries for what agents can and can’t do and controls how they execute their instructions in both open and closed models. The agent creator’s instructions become policy, detailing the files, credentials, and other areas the agent can access, and OpenShell verifies the limitations before the agents start their work and, with Nvidia’s Vera CPU (below), ensures they’re adhered to as the agents run their tasks.

OpenShell also can be used with chips from Intel and AMD.

The other key part of the Open Agent Safety Platform is Sentry, which extends the agent monitoring and instruction enforcement to Nvidia’s Bluefield-4 DPUs.

“Sentry provides in-silicon security enforcement, meaning that if an AI agent attempts to move outside its software boundary, Sentry quarantines and stops it in milliseconds,” the researchers wrote. “It combines threat detection, hardware-based agent governance and enforcement and data access protection from an isolated, out-of-band trust domain that is responsive in real time and invisible to agents and attackers.”

Sentry is built atop Nvidia’s DOCA, a software platform and framework the delivers the programmability to the Bluefield-4 DPU and links it with the OpenShell policy to create a record of an agent’s activities through its decisions, interactions, and the data and tools it accesses.

“This helps safety systems identify drift, investigate suspicious behavior, and determine when intervention or deeper analysis is needed,” they wrote. “The DOCA gateway complements this behavioral protection with identity governance, continuously verifying each agent’s identity and delegated authority to ensure it operates within its assigned scope.”

Nvidia co-founder and chief executive officer Jensen Huang said in a statement that the issue of AI security and safety is one that can be addressed through “full-stack engineering.” The new guardrail effort is being supported by dozens of companies, including AI vendors like Anthropic, Mistral, Tenex.AI, and SpaceXAI. Notably absent is OpenAI, which clearly needs to rein in its models.

Chief executive officer Sam Altman told CNBC his company is doing similar things, but disagreed that it’s simply an engineering issue.

“I don’t think it’s a full solution and I worry that if we treat AI safety as only an engineering problem, we will miss the very important point that we have a science problem in front of us,” Altman said. “We still have discovery of how to align AI models and we have to solve that scientific problem, too.”

SPONSORED FEATURE: HPE and NVIDIA help organizations apply accelerated computing, software, enterprise infrastructure, networking, control plane, and services to the workloads and business outcomes that matter most.

October 1, 2026

September 30, 2026

AMD Buys World Labs For $8.2 Billion To Get AI Model Cred