Understand this story
Anthropic has unveiled Anvil, an automated red-teaming system built on Claude that is designed to autonomously find vulnerabilities in model safety safeguards, and it has been tested in three domains: biosecurity, cybersecurity, and information security. In the information security tests, Anvil found its first substantive reasoning-information leakage vulnerability after sending 435 requests, whereas human experts required about 102,200 requests; in the 24 question-and-classifier combinations in biosecurity, the jailbreak schemes Anvil discovered in 23 combinations were more severe than those found by human testers. Anthropic also noted that Anvil's attack capability is correlated with the capability of the underlying Claude model, and that disabling the attack model's own safety safeguards is a key condition for it to achieve effective breakthroughs.
AI-generated from public reports
Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 13
ZhidongxiSelected
Anthropic Releases Automated Red-Teaming System Anvil, Finding Vulnerabilities in Information Security Testing with Only 1/235 of the Request Volume Used by Humans
Anthropic has unveiled Anvil, an automated red-teaming system built on Claude, designed to autonomously find vulnerabilities in model safety safeguards, and has conducted testing in three domains: biosecurity, cybersecurity, and information security. In information security testing, Anvil found its first substantive reasoning information leakage vulnerability after sending 435 requests, whereas human experts required approximately 102,200 requests; in 24 question-and-classifier combinations in biosecurity, Anvil discovered jailbreak schemes with higher severity than those found by human testers in 23 of the combinations. Anthropic also noted that Anvil's attack capability is related to the capability of the underlying Claude model, and that disabling the attack model's own safety safeguards is a key condition for it to achieve effective breakthroughs.
Phoenix Technology
Anthropic Releases Automated Red-Teaming System Anvil, Finding Vulnerabilities in Information Security Testing with Only 435 Requests
Anthropic announced Anvil, an automated red-teaming system built on Claude that autonomously seeks out vulnerabilities in model safety protections across three domains: biosecurity, cybersecurity, and information security. In biosecurity testing, Anvil found jailbreak schemes with higher severity than human testers in 23 of 24 question-and-classifier combinations; in information security testing, it found the first substantive reasoning information leakage vulnerability after sending only 435 requests, whereas human experts required approximately 102,200 requests. Anthropic also noted that Anvil's attack capability is positively correlated with the capability of the underlying Claude model, and that if the attacking model's own safety protections are enabled, 97% of red-team testing sessions would be blocked.
Heat trend for this story
- Current
- 5
- Period peak
- 13 Nov 14 04:00
- 24-hour change
- -50%
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.
AI Trends