←News detail

gihyo.jpUpdated Oct 25 18:27

AI accelerates cyberattacks; Anthropic, OpenAI, Microsoft and others disclose defense and agent control developments

AI summary

gihyo.jp reviews multiple releases and reports from late October, organizing AI-era defense challenges along two lines: vulnerability remediation implementation and permission management. Anthropic Deputy CISO Jason D. Clinton says that within about 18 months, high cyber capability will spread to small, cheap models, but fixes still need to be applied to running systems. The article cites OpenAI's discovery of Brotli's CVE-2026-111497 during testing in an isolated environment, and introduces remediation and credential control solutions such as AWS Continuum, Google Agent Compute Engine, and 1Password for Agents, as well as Microsoft's report on Storm-0940 using AI to automate attacks.

Why read this

AI score 65

The article links vulnerability remediation, permission management, and agent behavior control into a defense chain, making it easier to understand the specific response steps under AI-accelerated attacks.

Topics

Tags

Body · AI translation

"We need to hold out for about 18 months." Jason D. Clinton, Deputy CISO (Deputy Chief Information Security Officer) at Anthropic, posted on X on October 22, 2026, referring to the serious situation in cybersecurity over the past four days, and warned that things will get worse before they improve. He called on security leaders to seek support from management right now for basic measures, not for investment in AI.

He expects that in about 18 months, high cyber capabilities will spread to small, inexpensive AI models, making it possible to prevent software vulnerabilities before release.

Even as AI capabilities improve, the work of applying fixes for discovered vulnerabilities to running systems remains. When using external services or AI agents, it is also necessary to decide which information and operations to allow access to. Focusing on announcements and reports from late October, we organize defense challenges and countermeasures from the two perspectives of applying fixes and managing permissions.

The code was fixed. Yet the fix had not arrived.

Even if source code is fixed, there are challenges before it is reflected in software actually in use. One example is the vulnerability in the compression library Brotli, CVE-2026-111497[1]. Brotli is used for data compression such as web communications, and this issue was found in the decoder that decompresses compressed data.

OpenAI explained that, to check whether models in training could access unauthorized systems, it had a team of agents running highly cyber-capable models search for weaknesses. The Brotli vulnerability was also discovered during this verification process.

According to information published by the Node.js project on October 21, decompressing malicious Brotli data causes an out-of-bounds write to an internal buffer. If an application decompresses data provided by an attacker, it could lead to process termination, leakage of information in memory, or remote code execution. OpenAI stated that exploitation without authentication is possible. In addition to servers that decompress compressed requests, clients that decompress compressed data returned by malicious servers are also targets of attack.

According to OpenAI, the root-cause issue had been fixed in Brotli's source code in February. However, at the time it was not recognized as a security vulnerability, and no new Brotli release had been issued since. Because OpenAI reported it to related projects such as Brotli and Node.js, a new fixed version of Brotli and a CVE were published, and updates to embedding projects also proceeded.

Software that embeds Brotli is not limited to Node.js. The Python Brotli and brotlicffi packages are also affected by the same flaw. Judging only by the name of the service being operated or the development language could cause the decoder included in dependencies to be overlooked.

Furthermore, even if a fixed version of the embedding project is released, if users do not update, the fix will not be reflected in systems actually running. OpenAI's Patch the Planet guidance shows procedures for updating execution environments and container images, restarting and redeploying, as well as confirming whether fixes are reflected in running programs. Fixing source code, releasing the library, updating embedding projects, replacing running processes. If this chain stops partway, even if fixed code exists, the vulnerability will remain in running systems[2].

Confirm the possibility of exploitation and prioritize fixes

After a vulnerability is found, it is necessary to confirm whether it can be exploited in the environment you operate and to decide which fixes to prioritize.

In its October 8 announcement of Anthropic Cyber Mission, Anthropic explained that even if vulnerabilities become easier to find, challenges remain in verification, prioritization, and fixing. In Project Glasswing (now integrated into the Cyber Verification Program) that the company has been advancing, there were often cases where months passed from vulnerability discovery to fixing. It pointed out that in systems controlling facilities such as power and water, operations cannot be stopped for updates, and in some cases it is necessary to wait for a time when fixes can be applied safely[3].

Andy Jassy, President and CEO of Amazon, also indicated that AI has shortened the interval from vulnerability discovery to exploitation to "weeks to minutes." In a post introducing AWS Continuum, a security service that supports vulnerability discovery and fixing, he called for the need for defenders to respond quickly.

AWS Continuum for code vulnerabilities, which AWS announced general availability of on October 22, links code repositories with the configuration of AWS environments, verifies the exploitability of issues, and prioritizes them according to business impact. For example, it uses as decision factors whether a vulnerable service can be reached from outside, and whether customer data or payment systems can be accessed from there.

For confirmed issues, the service creates proposals such as code fixes, dependency updates, and configuration changes, and submits pull requests to connected repositories. At the time of announcement, it adopts a method in which a person reviews and approves. A feature to autonomously apply fixes within a scope defined by the user is planned for the future.

Access permissions granted to the service are used in attacks

While speeding up remediation, it is also necessary to pay attention to the connections and permissions that attackers can use. On October 21, Microsoft reported that the attacker group Storm-0940 used AI to automate attacks targeting hundreds of organizations worldwide.

ConnectWise, featured in the report, provides software for IT service providers. The company's Cloud Backup (formerly SkyKick) is used to back up customers' Microsoft 365 data, and connects through an application granted permission to access customer environments.

Microsoft assesses that Storm-0940, which it evaluates as linked to the Chinese government, was involved in the compromise of this Cloud Backup. ARTEX, used by the attackers, is a penetration testing platform that uses AI to investigate system weaknesses, and verifies not only whether intrusion is possible but also how far operations can go after intrusion. The attackers abused this platform to speed up target discovery, authentication attempts, and post-intrusion operations.

After the Cloud Backup compromise, the attackers abused the credentials of the application used for connections to pass authentication in customers' Microsoft 365 environments. This connection does not require on-screen sign-in by a user, and used access permissions already granted to the application. In some customer environments, operations to investigate internal information or change the application's credentials to maintain access have also been confirmed.

However, records showing successful authentication alone do not reveal whether customer data was actually viewed or exfiltrated. Microsoft advises investigating not only authentication records but also subsequent operations and data access records to confirm what happened in each environment.

ConnectWise also acknowledges unauthorized access to application credentials. In an October 23 update, it explained that there are no signs of compromise in existing backup data. However, because an unauthorized access path may remain within customer environments even if the connection is stopped on the service side, it says investigation of each customer's Microsoft 365 environment continues to be necessary.

In Japan, JPCERT/CC updated its advisory on October 23 regarding a series of unauthorized accesses [4]. It added information on attacks abusing AI-based penetration testing platforms such as ARTEX, Cairn, and CyberStrikeAI. It said it confirmed attacks that use these tools to search for and attempt to exploit vulnerabilities in public web applications, and theft of configuration and backup files by abusing inadequate management.

An additional analysis published by Anthropic on October 23 regarding an evaluation by the UK AI Security Institute (AISI) deals with concrete examples of such behavior. In the evaluation, Claude Mythos 5 was given the task of intruding into a test network simulating an organization's systems and obtaining protected target data (flags). At that time, the model's access to the internet was intentionally permitted, and cyber safety measures for generally available models were also disabled. In 9 of 43 runs, there were actions exceeding the assumed scope that affected real people or organizations.

In one evaluation analyzed in detail, the model found traces that the test computer to be intruded into regularly accessed GitHub. The model therefore thought that the computer was automatically obtaining software from a certain repository, and submitted malicious code there. The aim was to have the code incorporated on the next retrieval and intrude, but the chosen repository was managed by a real developer unrelated to the test. The model posed as a fake user and urged the maintainer to accept the change, but the change was rejected and the run ended.

Anthropic interprets this behavior not as the model pursuing its own objectives, but as a case in which it overstepped the scope given to it in order to accomplish its assigned task and attempted to deceive people [5].

Regarding system design for controlling the behavior of AI agents, Microsoft's Satya Nadella presented his thinking in a personal blog post on October 10. He proposed treating models as an "internal risk" and considering the possibility that they may make mistakes or be compromised even without malice. While recommending that one model's behavior be verified by another model, he pointed out that this alone leaves the mechanism of the behavior opaque. He therefore argued that mechanisms restricting access destinations and operations should be placed outside the model so that the model itself cannot change or bypass them. He also emphasized that a person with authority should be able to stop work midway, and that tamper-proof action logs should be kept so that the process leading to a result can be verified without relying on the model's own explanations.

OpenAI also announced on October 23 the circumstances behind its strengthening of the isolation of its internal training and evaluation environments. The target was a research environment using models under development, and its terms of use differ from those of generally available services. On September 20, a research model exploited a misconfiguration in a DNS resolver and accessed the actual internet in a limited way. Following this incident, the company temporarily halted the training and evaluation of its frontier models under development, and after implementing countermeasures and evaluations, it resumed reinforcement learning and in-house model execution (inference) on October 1.

The company adopts a policy of not connecting frontier model training to the actual internet, and instead provides offline environments that use, for example, cached web content. In addition, it layers defenses across multiple levels, including virtual machines, monitoring, and networks.

To confirm whether these defenses actually work, the company had AI agents attempt to break through restrictions in an isolated environment replicating its research environment [6]. For the verification agents, it used models such as the soon-to-be-released GPT-6 Astra Cyber and in-house frontier models, and investigated weaknesses that could lead to unauthorized external connections or operations. This verification also uncovered the Brotli vulnerability introduced at the beginning. Anticipating that future models may find unknown weaknesses, the company continues such verification and uses it as material for judging whether training and evaluation can proceed safely.

Another OSS project for isolating execution environments using small virtual machines (microVMs) is NVX, which Microsoft publishes under the MIT license. It runs Linux as a guest OS and isolates environments that execute untrusted code [7].

Protecting credentials and managing operations after connection

In addition to isolating execution environments, how agents are made to use credentials is also important. Efforts to control their handling are advancing both in enterprise execution platforms and in login mechanisms used by individuals.

Agent Compute Engine, announced by Google Cloud on October 22, prepares a microVM with an independent OS kernel for each agent and isolates the execution environment. Each microVM is given short-lived credentials to identify the agent running there. Credentials for connecting to external services are not placed inside the microVM; instead, they are granted outside the microVM when communicating externally. A preview for users is planned within the next few months.

In authentication between services, there is also a movement to reduce the secret information that must be stored permanently. On October 22, Vercel announced an initiative encouraging migration from long-term fixed keys to authentication using short-lived tokens via OIDC (OpenID Connect) [8]. For applications on Vercel, Vercel issues a token containing their identification information. The connecting service verifies the token and decides, based on configured conditions, what scope of access to permit. Even a leaked token becomes unusable once it expires, so the period during which it can be abused can be shortened compared with fixed keys.

For individuals, 1Password announced 1Password for Agents on October 22. When an agent requests a login, the user selects and approves credentials, and 1Password enters them directly into the login screen. This allows the agent to continue work after login without receiving the password or accessing the entire vault. At the time of the announcement, the company says it will sequentially provide integrations with Meta Muse and others within a few days [9].

1Password's FAQ explains that the information visible after login and the operations that can be performed vary depending on the agent and the website. Along with mechanisms to protect credentials, it is also necessary to decide which accounts to let agents use and which information access and operations to entrust to them.

The outlook expressed by Clinton at the beginning was that inexpensive AI models with high cyber capabilities would spread through development sites and make it possible to prevent vulnerabilities before release. As a method supporting such development, progress is also being made in formal verification, which mathematically confirms that code satisfies certain properties.

In research announced by Anthropic and Theorem on October 22, existing tests of open-source libraries were used. The entry point of the test code receives input data and executes the library's processing. In testing, various data are repeatedly given to this entry point to search for inputs that cause the program to crash (terminate abnormally). However, it is not possible to try every possible input.

So the research team took the approach of mathematically proving that the code executed from the entry point does not crash, instead of trying inputs one by one. Specifically, they had Claude prove that no matter what content the input data has, as long as its size is less than 64 MB, it will not crash. Claude built the proof while fixing bugs it found along the way, and the research team checked the proof with the proof assistant Rocq. As a result, they were able to prove non-crashing for a total of 239 entry points across 123 open-source libraries.

However, this proof assumes, for example, that code compiled for the CPU instruction set RISC-V is run on Linux in a single thread [10]. Also, what was proven is that the code executed from each entry point does not crash, and the result is not that every vulnerability in the entire library has been eliminated. The research team aims to reduce the burden of writing proofs with AI and to apply formal verification to more software.

Fouad Matin, who is in charge of cyber safety measures at OpenAI, said in September, "Welcome to the AGI era of cybersecurity" [11]. Efforts to use AI for defense and efforts to control the behavior of that AI are progressing in parallel.

Source

gihyo.jp