AI Trends

New Google DeepMind paper: swarms of agentic attackers can still break the most robustly defended models

Rohan PaulUpdated Nov 13 23:27

AI summary

A new Google DeepMind paper shows that prompt injection remains unsolved: an Agentic Optimizer composed of multiple AI attack agents can still fool the most robustly defended models. The system consists of 8 groups of AI agents that test injections, share notes, and iterate on each other's best attempts, tested on 100 tool-calling tasks, with the attacker seeking only to trigger a single random, harmless tool call. At up to 320 attempts, the strongest attacker breached 34% of tasks on the most robust Gemini 4 Argon, while the open-weight models Kimi K3, GLM-5.3, and DeepSeek-V4.1-Flash withstood only 1% of tasks.

Why read this

AI score 73

The paper uses multi-agent coordinated attacks to test prompt injection, providing a comparison of specific success rates between the strongest attacker and each model.

Topics

Tags

Body

New Google DeepMind paper shows prompt injection is not solved, since a swarm of strong AI attacker agents still tricks even the best-defended models.

Prompt injection means hiding instructions in data an AI agent reads, like a tool result, to make it act for the attacker. Human red-teaming attacks now almost never work on recent Gemini models, so the problem can look solved.

They built Agentic Optimizer, 8 teams of AI agents that test injections, share notes, and build on each other's best attempts. They tested it on 100 tool-use tasks where the attacker only wanted one random, harmless tool call.

With up to 320 tries, the strongest attacker beat Gemini 4 Argon, the most robust model, on 34% of tasks. Open-weight Kimi K3, GLM-5.3, and DeepSeek-V4.1-Flash resisted only 1% of tasks.

Source

Rohan Paul