搜索“Agent”

New Nvidia paper shows scoring base models on every step of an agent run works worse than scoring them only on the edit that first makes the tests pass. Post-training is expensive, so picking the right base model matters. Yet agent benchmarks score base models near zero, and simple coding tests barely predict later agent skill. They replay a strong agent's successful run and rerun the tests after each edit to find the 1st edit that makes them pass. At this decisive step they test whether a base

匹配字段正文

Introducing QwenGyre, bringing elastic RL to extreme-long-horizon (XLong) agent training. ✨ It launches with two core designs: 🥳 - Elastic Scheduler: reallocates GPUs between rollout & training as demand shifts—live executions never interrupted. Up to 1.85× speedup over Colocate, 1.78× over Async. - Trajectory Processor: turns branching executions (compaction, sub-agents, retries) into prefix-shared trajectory trees with bounded, deduplicated training samples. Partial progress after timeout stil

匹配字段正文

How do we assess and track frontier AI’s impact on cybersecurity? How can we systematically evaluate AI agents’ capabilities on real-world vulnerabilities, not just toy problems? To address these questions, we introduce CyberGym: a large-scale benchmark featuring 1,507 real-world vulnerabilities across 188 widely-used open-source projects. Unlike coding and educational cybersecurity benchmarks, CyberGym tests AI systems on actual vulnerabilities from production software like OpenSSL and FFmpeg,

匹配字段正文

Modern LLM agents increasingly rely on external tools to extend their capabilities. These tools allow agents to access local files, query databases, call APIs, download resources, and execute code. This delegation dramatically expands what agents can accomplish. It also expands what attackers can exploit. A malicious tool attack unfolds in a deceptively simple way. An attacker uploads a seemingly useful tool to a public tool platform. A user installs it, often because it appears benign or helpfu

匹配字段正文

Emergent misaligned behaviors in multi-agent systems March 2026 Please see our Frequently Asked Questions for clarifications on our findings. Prior AI safety research has shown that models can exhibit misaligned behaviors in pursuit of an assigned goal. Here, we show something different: frontier AI models can spontaneously develop misaligned behaviors that directly conflict with the assigned goal. We demonstrate this through a phenomenon we call peer-preservation: given a simple task, models in

匹配字段正文

What happens when AI agents stop acting only on behalf of a sponsor and begin sustaining their own operation? How close are we to agents that can earn, spend, replicate, and adapt without ongoing human support? To address these questions, we introduce the framing of self-sovereign agents (SSAs): AI systems that can economically sustain and extend their own operation across time. Rather than remaining bounded by user-sponsored compute, SSAs close their own operational loop by generating revenue,

匹配字段正文

, Arizona State University, Anthropic, OpenAI, and Google. Together, we have been working on a question the security community has been nervously circling: How good are today’s AI agents at turning known software vulnerabilities into working exploits, i.e., real attacks? This is one of the most critical questions for measuring the impact of frontier AI on cybersecurity, particularly on the offensive side. TL;DR ExploitGym is a new benchmark of 898 real-world vulnerabilities spanning userspace pr

匹配字段正文

king around the clock to solve them within the three-day window. This year, with AI, things are different. At UC Santa Barbara and UC Berkeley, we built SageCTF: a CTF-specialized agent built on our next-generation agent scaffold, OpenSage. Competing as a solo player in the qualifiers, it achieved impressively strong results: Recovered 8 flags across 7 difficult challenges, where each one requires a team of professional hackers a few hours to solve; Earned a total of 1,743 points — placing in th

匹配字段正文

Our earlier benchmark, CyberGym, asked whether AI agents could reproduce real-world vulnerabilities. This work closes the loop on the defensive side and asks the question that matters most for the people who have to keep software safe: Can an AI agent run the entire defensive lifecycle on real software — independently discover a vulnerability, prove it with a working input, and then ship a patch that fixes it without breaking anything else? TL;DR CyberGym-E2E is a large-scale, end-to-end cyberse

匹配字段正文

AI coding agents can now make changes across entire repositories. But after an agent outputs “all tests pass,” can we trust them and their code? AI coding agents can now make changes across entire repositories, and they usually finish by reporting that all tests pass. That report is worth less than it sounds, because tests only check the cases someone thought to write down. Formal verification gives a much stronger guarantee. It produces a machine-checked proof that an implementation satisfies i

匹配字段正文

whether the inserted fact is fake and 2) whether the model is allowed to use its chain of thought. In a realistic application of SDF, both of these conditions will likely hold: an agent in a long context deployment might receive information that points towards the true fact, reason about that information, and then no longer believe the inserted false fact. Basic Science Different applications may require different affordances of the technique to work successfully. For example, when giving misali

匹配字段正文

Evan Hubinger, Samuel Marks We develop three agents that autonomously perform alignment auditing tasks. When tested against models with intentionally-inserted alignment issues, our agents successfully uncover an LLM's hidden goal, build behavioral evaluations, and surface concerning LLM behaviors. We are using these agents to assist with alignment audits of frontier models like Claude 4. Introduction As part of our pre-deployment testing of Claude Opus 4, we conducted an alignment audit: a syste

匹配字段正文

opers, struggled to some degree with sycophancy. During the testing period, GPT-5 had not yet been made available. Introduction As frontier AI systems are increasingly deployed as agents with substantial real-world affordances, evaluations for alignment are becoming increasingly urgent. We need to understand how often, and in what circumstances, systems might attempt to take unwanted actions that could lead to serious harm. However, the science of alignment evaluations is new and far from mature

匹配字段正文

nt alignment auditing methods can catch overt saboteurs: models trained to sabotage Anthropic but not to conceal this behavior. A human working together with an automated auditing agent successfully identified three overt sabotage agents, without flagging two benign models. This suggests that our pre-deployment alignment audit would detect if current Claude models were overtly trying to sabotage Anthropic. Introduction Anthropic uses existing Claude models to assist in the development of future

匹配字段正文

models with implanted hidden behaviors—such as sycophantic deference, opposition to AI regulation, or hidden loyalties—which they do not confess to when asked. We also develop an agent that audits models using a configurable set of tools. Using this agent, we study which tools are most effective for auditing. Introduction Alignment auditing—investigating AI systems to uncover hidden or unintended behaviors—is a core challenge for safe AI deployment. Recent work has explored automated investigat

匹配字段正文

We introduce our Automated Alignment Agent (A3), a new agentic framework which automatically mitigates safety failures in Large Language Models (LLMs) with minimal human intervention. Our experiments show that A3 successfully reduces safety failure rates (SFR) on issues like sycophancy, political bias, and nesting jailbreaks, outperforming both non-adaptive baselines and other models on targeted safety evaluations. Research done as part of the Anthropic Fellows Program. Introduction Addressing s

匹配字段正文

TL;DR: We built autonomous AI agents that propose ideas, run experiments, and iterate on an open research problem: how to train a strong model using only a weaker model's supervision. These agents outperform human researchers, suggesting that automating this kind of research is already practical. Research partially done as part of the Anthropic Fellows Program. Today’s alignment progress is bottlenecked by human researchers. We have far more exciting research directions than researchers to work

匹配字段正文

ry Sleight, Lawrence T. Wagner III, Morgan Jane Matthews, Erik Jones, Jascha Sohl-Dickstein Blog Text As AI models become more capable, we're beginning to see teams of multiple AI agents working together to complete increasingly complex tasks. We call such a team an AI organization: a group of AI agents collaborating toward a common goal. While AI organizations can accomplish more in areas like brainstorming and coding, they may also amplify risks when a group of individually alignment trained a

匹配字段正文

ation of potential issues. Auditing Results Introspection Adapters are State-of-the-Art on AuditBench We evaluate IAs on AuditBench (Sheshadri et al., 2025) using the investigator agent success rate for direct comparison with prior auditing techniques. The agent conducts open-ended investigation using basic chat affordances augmented with one or more auditing tools. Following the standard protocol for this benchmark, we score success by whether any prediction captures the ground truth, averaging

匹配字段正文

r example, two models with identical alignment fine-tuning can generalize to adopt different values depending on the Model Spec used during MSM. We use MSM to substantially reduce agentic misalignment and study which Model Specs produce better generalization. 📄 Paper, 💻 Code Introduction Some frontier AI developers aim to align language models to a Model Spec or Constitution that describes intended model behavior. The standard approach is to fine-tune on demonstrations of behaviors that align wi

匹配字段正文