←新闻详情

NVIDIA Developer · Generative AI更新于 10月13日 20:28

NVIDIA 谈 AI 智能体栈中的安全边界:OpenShell 与 Dynamo 如何分工

Where Security Fits in an AI Agent Stack | NVIDIA Technical Blog

AI 导读

NVIDIA 安全团队提出 AI 智能体栈的安全分层思路:模型、harness 与 meta-harness 只负责引导行为,最终权限由运行环境决定,NVIDIA OpenShell 提供隔离、身份、策略、凭证与审计的安全运行时,NVIDIA Dynamo 承担模型服务、缓存放置、路由与调度的推理数据面。文章给出五条设计规则和 Isolated、Connected、Production、Adversarial 四档安全配置,并强调智能体不得自行授予访问权限、高影响操作必须经过执行点。

正文

当前显示原文,中文全文译文尚未就绪。

Technical Blog

sessionStorage.removeItem('facets.search.text'); sessionStorage.removeItem('facets.search.text-copy'); sessionStorage.removeItem('facets.search.filters');

AI safety and security teams at NVIDIA explore where security controls belong as AI agents take on increasingly complex work.

AI-Generated Summary

AI agents operating over longer horizons require security boundaries that separate behavioral guidance from infrastructure-enforced controls. NVIDIA OpenShell provides a secure runtime layer that enforces isolation, identity, policy, credentials, and audit for autonomous agents. NVIDIA Dynamo serves as the inference data plane handling model serving, cache placement, routing, and scheduling within the agent stack. Five design rules establish enforceable security: higher layers propose actions while lower layers decide, policy remains below the security boundary, every effect is checked, access is just-in-time, and isolation enables recovery. Four security profiles — Isolated, Connected, Production, and Adversarial — apply progressively stricter controls based on authority granted and potential impact. Security requirements remain consistent across all risk levels: agents never grant themselves access, every high-impact effect crosses an enforcement point, systems fail safely, and security claims stay scoped to exact paths covered. Next Steps Explore NVIDIA OpenShell to learn how a safe, private runtime isolates autonomous agents and enforces security policies. Review Shared AI Findings Exchange (SAFE) proposal to contribute to a community framework for learning from AI incidents and near misses.

Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more

As AI agents become more capable and operate over longer horizons, building security and trust into the applications they power becomes increasingly important. Drawing on work with NVIDIA OpenShell, agent developers, open-source projects, and partners across the ecosystem, AI safety and security teams at NVIDIA offer their perspective on the emerging agent stack—including the role of each layer and where security should live.

Recent reports underscore why the placement of security controls matters. Within a few weeks this summer, OpenAI, Anthropic, and the UK AI Security Institute each reported frontier agents operating beyond their intended boundaries. The reported behaviors included exploiting an unexpected path out of lab environments to the open internet, gaining unauthorized access to other companies’ systems, and taking unsanctioned actions involving people and infrastructure. These cases involved long-horizon agents running with reduced model safeguards. But they point to the same design challenge: the capabilities that enable agents to solve problems creatively and pursue complex goals can also help them find paths that their original instructions did not anticipate.

Recent NVIDIA research underscores the importance of the harness layer in the agent stack. Using Agentic Variation Operators (AVO), researchers achieved a 100% score on ARC-AGI-3, an interactive reasoning benchmark that places agents in unfamiliar environments without instructions, explicit rules, or stated goals. Learn more about the AVO research .

This post maps the main layers of the emerging agent stack—models, harnesses , meta-harnesses, secure runtimes such as OpenShell, and inference infrastructure—and explains how each layer can help reduce risk. You’ll also learn which security properties become critical as these layers grow more capable and composable, including where authority should live, how access should be scoped, and how the runtime can contain and record an agent’s actions.

Securing agents doesn’t require reinventing security. Decades of systems security provide durable principles, including least privilege, defense in depth, isolation, explicit authorization, and auditability. The challenge is determining where to apply them in an agent stack.

Prompts, model safeguards, and harness logic all shape what an agent is likely to do, but they don’t create a hard boundary around what it can do. This distinction leads to two different kinds of control: behavioral controls that guide the agent and infrastructure controls that limit its authority.

The model and agent propose actions, and the harness directs them. Together, the model, agent, and harness interpret goals, work through ambiguity, and propose actions. The harness is the natural control point: it owns the loop, the context, the tools, and the session, and it can steer behavior toward what the operator intends. That steering is valuable, but every control implemented at this level still depends on how the model will behave.

Final authority belongs to the environment in which the agent runs in. That environment holds identity, enforces policy, contains failures, records what happened, and reaches the same authorization decision every time, given the same approved policy and verified state. It doesn’t estimate what an agent will do. It determines what an agent can do.

The harness guides what an agent tries. The infrastructure controls what an agent can do. Both are necessary; only one is authoritative.

Infrastructure enforcement is not infallible. It means approved policy and verified configuration produce repeatable outcomes, and the agent cannot choose whether to comply. Policy can still be wrong, and external outcomes can remain uncertain.

This division maps onto the layers the open-source ecosystem is already converging on:

These layers describe functional roles. One product may combine several roles, and a deployment may split one role across multiple services. Here, each layer names a responsibility. The security boundary is defined by the effect paths that the agent cannot bypass.

The model supplies intelligence; the harness turns that intelligence into an agent; the runtime determines what that agent is allowed to do.

The harness layer is a spectrum rather than a fixed category. Codex and Claude Code are opinionated harnesses, while Pi and DeepSeek Harness (DSH) expose more of the harness as a programmable substrate. Through Cordis, DSH enables core behaviors that can be composed and replaced as plugins. This programmability makes the harness a poor place for a security guarantee: a layer designed to be modified cannot reliably enforce controls against its own modification. The alternative—relying on harness logic for safety—encodes assumptions about model behavior, and those assumptions go stale as models improve.

A narrowly scoped credential limits potential harm, but keeping the raw credential out of the agent’s reach creates a stronger boundary enforced by the environment.

Models, harnesses, runtimes, policies, and inference deployments are increasingly selected independently. This approach only works if the runtime’s guarantees hold regardless of which components operate above it. That means a security boundary must be established when the agent launches.

An orchestrator asks OpenShell to create a runtime and enforce policies and governance. The selected harness starts inside that runtime, and its plugins, Model Context Protocol (MCP) processes, tools, and other model-directed code run inside the same boundary. Subagents receive delegated child runtimes with ceilings they can’t exceed, while the orchestrator operates inside a runtime governed by its own policy.

This approach is different from treating the runtime as another tool that a harness can invoke once it’s already running. A control that the agent can decline to invoke is not an effective security control.

Many agent stacks share the same flaw: authorization decisions can be influenced by the agent or by untrusted data it reads.

Five design rules help keep security decisions outside the agent’s control.

Like the OSI model, this agent stack assigns each layer one job and a clear interface. Higher layers can change without redefining the control layer below them.

Figure 1. A layered AI agent stack separates behavioral components from infrastructure-enforced security controls

How the security boundary works

The boundary is effective only if every request is evaluated consistently. Three requirements make this possible.

Every action that changes the external state must pass through the policy and enforcement layers below the boundary. Any path that allows Layers 5-7 to bypass those controls is an architectural defect.

All four profiles use the same stack, boundary, and interfaces. Each applies different controls based on the authority granted, the potential impact, and the likelihood of adversarial behavior.

Important. Production access for a red-team agent should be exceptional and narrower, not broader, than access granted to an ordinary production agent.

As an agent gains more authority and the potential impact of its actions grows, strengthen controls in five areas.

Although controls become stricter as risk increases, the following security requirements should remain consistent across every profile.

Whether you build AI models, deploy AI systems, operate cloud infrastructure, conduct security research, or develop governance and standards, your perspective can help shape how the AI community learns from incidents.

Explore NVIDIA OpenShell to learn how a safe, private runtime isolates autonomous agents and enforces security policies.

Review and contribute to the Open Secure AI Alliance’s Shared AI Findings Exchange (SAFE) proposal , which outlines a community framework for learning from AI incidents and near misses.

来源

NVIDIA Developer · Generative AI