Google GTIG report says AI framework can now turn security patches into exploits with no human in the loop
Google's Threat Intelligence Group (GTIG) said in a quarterly report that it observed for the first time attackers using a no-human-in-the-loop AI framework, exploit_agent, to turn security patches into executable exploits. The framework was deployed on an exposed staging server, connected to multiple frontier models and locally hosted large language models, and continuously compared the code of popular web applications before and after patch releases; after identifying suspected flaws, it automatically executed the generated exploit code. The report also noted that the same server held victim data as well as scripts for brute-forcing and installing webshells, and that victim telemetry showed the operator mainly targeted French media, entertainment, and education websites.
Why read this
SelectedAI score 81
The report provides an example of a no-human-in-the-loop exploit framework and contrasts it with the previous quarter, when no fully autonomous attack chain had yet been observed.
Topics
Body
A new report out today from Google LLC’s Google Threat Intelligence Group finds that for the first time, attackers have used an artificial intelligence framework running with no human in the loop to turn security patches into exploits.
Called exploit_agent, the framework turned up on an exposed staging server, where it was wired into several frontier and locally hosted large language models. Its job, according to GTIG, is continuous comparison of popular web application code before and after each patch announcement. Once a likely flaw is identified, the exploit it drafts is executed automatically.
Data taken from victims was sitting on the same server, alongside scripts for brute-forcing, installing webshells and even removing webshells left by rival attackers. Victim telemetry suggests the operator was most interested in French websites tied to media, entertainment and education. Written in Russian, one comment in the framework’s configuration file tells whoever set it up to enter the keys “right here, not in the chat.”
In the previous edition of the quarterly tracker two months ago, GTIG said it had not observed fully autonomous attack pipelines deployed against targets in the wild. That changed over the past quarter, according to the new report, titled “Agentic Threat Activity Crosses a Significant Threshold.” Because the source material for a patched flaw is finite, GTIG believes attackers may find it more efficient to point LLMs at n-day vulnerabilities than to hunt for zero-days.
How quickly the gap between disclosure and a working exploit can close shows in the case of two flaws in print management servers from PaperCut Software Pty Ltd., chained together and exploited as zero-days in late August. Security researchers and Huntress Labs Inc. used large language models to generate working exploits within a day of disclosure.
A campaign tracked by GreyNoise Intelligence Inc. four days after disclosure is believed to have relied on LLMs to build its exploits before it hit hundreds of organizations. One foothold in that campaign reached domain administrator within minutes, GreyNoise found.
Elsewhere, open-source agentic penetration testing frameworks are being used to automate discrete steps of live intrusions. During one intrusion against the U.S. aerospace sector, a threat actor was observed using the multi-agent framework CAIRN.
Another actor ran a customized platform built on PentAGI between late August and early October to scan a wide range of organizations in Asia for vulnerabilities. For now, the noisy forensic trail left by unrestricted autonomous execution is said to make the frameworks easier to detect.
According to GTIG, influence operations now use models for far more than writing posts. Using cloud phone farms, a turnkey platform that one commercial contractor was building with help from the Gemini API is designed to run more than 100,000 social media accounts. To make generated photos pass as smartphone shots, prompts behind a separate platform, which targeted political sentiment across Europe and the Middle East, are said to have called for deliberate camera flaws.
Among government-backed actors, photos on the fake recruiter profiles that a suspected North Korean group tracked as UNC6836 used to approach cryptocurrency organizations in North America and Europe were found to carry Google’s SynthID watermarks. By way of altered .claude/settings.json files in internal code repositories, the group later rigged each launch of Anthropic PBC’s Claude Code to download its SOMBERMEME backdoor.
Google said it disabled the accounts and assets tied to the Gemini misuse it found. Google DeepMind has also used the findings to strengthen its classifiers.
Image: SiliconANGLE/GPT Image 2.6