AI Trends

Trending story

Anthropic unveils automated red-teaming system Anvil

2 reports2 story updatesUpdated Nov 13 12:16

Understand this story

AI overview

Anthropic has unveiled Anvil, an automated red-teaming system built on Claude that is designed to autonomously find vulnerabilities in model safety safeguards, and it has been tested in three domains: biosecurity, cybersecurity, and information security. In the information security tests, Anvil found its first substantive reasoning-information leakage vulnerability after sending 435 requests, whereas human experts required about 102,200 requests; in the 24 question-and-classifier combinations in biosecurity, the jailbreak schemes Anvil discovered in 23 combinations were more severe than those found by human testers. Anthropic also noted that Anvil's attack capability is correlated with the capability of the underlying Claude model, and that disabling the attack model's own safety safeguards is a key condition for it to achieve effective breakthroughs.

AI-generated from public reports

Latest development · Report

Anthropic Releases Automated Red-Teaming System Anvil, Finding Vulnerabilities in Information Security Testing with Only 1/235 of the Request Volume Used by Humans

Developments

1 confirmed developments

  1. Anthropic unveils automated red team testing system Anvil

    Source:Zhidx

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 13

  1. ZhidongxiSelected

    Anthropic Releases Automated Red-Teaming System Anvil, Finding Vulnerabilities in Information Security Testing with Only 1/235 of the Request Volume Used by Humans

    Anthropic has unveiled Anvil, an automated red-teaming system built on Claude, designed to autonomously find vulnerabilities in model safety safeguards, and has conducted testing in three domains: biosecurity, cybersecurity, and information security. In information security testing, Anvil found its first substantive reasoning information leakage vulnerability after sending 435 requests, whereas human experts required approximately 102,200 requests; in 24 question-and-classifier combinations in biosecurity, Anvil discovered jailbreak schemes with higher severity than those found by human testers in 23 of the combinations. Anthropic also noted that Anvil's attack capability is related to the capability of the underlying Claude model, and that disabling the attack model's own safety safeguards is a key condition for it to achieve effective breakthroughs.

  2. Phoenix Technology

    Anthropic Releases Automated Red-Teaming System Anvil, Finding Vulnerabilities in Information Security Testing with Only 435 Requests

    Anthropic announced Anvil, an automated red-teaming system built on Claude that autonomously seeks out vulnerabilities in model safety protections across three domains: biosecurity, cybersecurity, and information security. In biosecurity testing, Anvil found jailbreak schemes with higher severity than human testers in 23 of 24 question-and-classifier combinations; in information security testing, it found the first substantive reasoning information leakage vulnerability after sending only 435 requests, whereas human experts required approximately 102,200 requests. Anthropic also noted that Anvil's attack capability is positively correlated with the capability of the underlying Claude model, and that if the attacking model's own safety protections are enabled, 97% of red-team testing sessions would be blocked.

Heat trend for this story

Current
5
Period peak
13 Nov 14 04:00
24-hour change
-50%

Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.

Related stories

  1. Related storiesGoogle DeepMind releases the Gemini Robotics 2.5 model
  2. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  3. Related storiesOpenAI releases GPT-6.2 Astra and Sol, ARC Prize announces perfect scores
  4. Related storiesMustafa Suleyman launches MAI Code 2 and MAI Code 2 Flash