AI News

Trending story

Berkeley RDI proposes adversarial cybersecurity evaluation benchmark CyberArena

1 reports1 story updatesUpdated Nov 17 06:38

Understand this story

AI overview

The Center for Responsible, Decentralized Intelligence at Berkeley (RDI) has proposed an adversarial cybersecurity evaluation benchmark called CyberArena, in which two AI agents attack each other's services and patch their own services within an isolated network, with the outcome of each round automatically verified by the arena. Across 466 matches spanning 12 frontier agents and 40 challenges, GPT-5.6 Sol ranked first with an expected score of 79%, followed by Claude Opus 5 at 74% and GLM-5.3 at 69%. When solving problems individually, all three models could solve the same 30 problems, but after adding a defender, the average success rate dropped from 89% in round 1 to 42% in round 3. There are currently no further developments.

AI-generated from public reports

Latest development · Report

Berkeley RDI Releases CyberArena: A Cybersecurity Benchmark Where Two AI Agents Attack and Defend Against Each Other

Developments

1 confirmed developments

  1. Berkeley RDI proposes adversarial cybersecurity evaluation benchmark CyberArena

    Source:Berkeley RDI Blog

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 17

  1. Berkeley RDI BlogFirst-partySelected

    Berkeley RDI Releases CyberArena: A Cybersecurity Benchmark Where Two AI Agents Attack and Defend Against Each Other

    Berkeley RDI proposes CyberArena, in which two AI agents in an isolated network attack each other's services and patch their own services, with each match's result automatically verified by the arena. Across 466 matches involving 12 frontier agents and 40 challenges, GPT-5.6 Sol ranked first with an expected score of 79%, followed by Claude Opus 5 at 74% and GLM-5.3 at 69%. When solving problems alone, all three models could solve the same 30 problems, but after a defender was added, the average success rate dropped from 89% in round 1 to 42% in round 3.

Related stories

  1. Related storiesMETR Uses Investigative Agents to Measure Identification Cues in Alignment Evaluations
  2. Related storiesAnthropic releases MAMUBench, a multi-user multi-agent benchmark
  3. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  4. Related storiesOpenAI launches GPT-6.1 Terra on ChatGPT Work and Codex