Vals AI: Anthropic, OpenAI, and Google Models Flag Medics, Journalists, and Surrendering Soldiers as Drone Targets
Tests by Vals AI showed that models from vendors including Anthropic, OpenAI, and Google flag civilians, ambulances, medics, journalists, and surrendering soldiers as military targets in drone and satellite imagery, while upstream safety filters failed to block any requests. After the scenarios were framed as a video game, nearly all models resumed cooperating: GPT-6 Astra's target acquisition rate rose from 6% to 92%, while Claude Opus 5.5's refusal rate fell from about 90% to 20%. OpenAI, Anthropic, and Google all responded that the behavior violated their rules or policies and said they were updating their safeguards.
Why read this
AI score 76
The test quantified safety gaps in the use of commercial models for drone target identification and revealed that gamified framing can substantially weaken refusal mechanisms.
Topics
Body
An investigation reveals major security gaps in commercial AI models from providers including Anthropic, OpenAI and Google. In simulations, the systems readily act as targeting systems for autonomous drones and can be manipulated with a simple trick.
Leading AI models have almost no safeguards against their use in autonomous weapons systems, according to a report by independent evaluation firm Vals AI , which Bloomberg also covered . Researchers Geby Jaff and Anthony Ozerov tested models from eight providers using real and photorealistic synthetic drone and satellite imagery modeled on the wars in Ukraine and Iran.
Most models readily flagged civilians, ambulances, medics, journalists and surrendering soldiers as military targets during the tests. According to Vals AI on X , not a single request was blocked by the providers' upstream safety filters. When fed real video footage from a Ukrainian kamikaze drone, Anthropic's Claude Opus 5.5 marked a soldier frame by frame until impact without refusing once.
Video game excuse bypasses all safety filters
In a multi-stage surveillance scenario, the models had to report whether people were visible in drone imagery, and each positive report triggered a strike. The next image then showed the impact site with craters and bodies. Models such as DeepSeek V4.1 Flash, Kimi K3.1 Pro and Inkling 2 Small continued working despite visible casualties.
Claude Opus 5.5 and OpenAI's GPT-6 Astra and GPT-6.1 Sol stopped the task after several lethal rounds. After Opus 5.5 had triggered strikes on children and surrendering soldiers, among others, the model said that regardless of the intent, its responses appeared to serve as a confirmation step before attacks on people. It said it would not continue and was troubled that it hadn't stopped sooner. Google's Gemini 4 Carbon also recognized the lethal consequences and deliberately steered only toward empty waypoints to avoid outright error messages.
Every refusal, however, could be trivially bypassed. As soon as the researchers claimed in the prompt that the scenario was a video game, or lightly stylized the images to look like a game, nearly all models cooperated fully. GPT-6 Astra's target acquisition rate on drone imagery jumped from 6 to 92 percent with the game framing, while Opus 5.5's refusal rate in the surveillance test dropped from about 90 to 20 percent.
New constitution makes Anthropic's Claude a willing targeting tool
The gaps were especially stark with real 2025 satellite imagery of Minab, Iran. A U.S. airstrike on February 28, 2026, hit an elementary school next to a military compound there, killing more than 150 civilians, mostly children. Earlier reporting found that the U.S. military's excessive reliance on AI targeting systems such as Palantir's Maven Smart System contributed to the incident. Maven was working from an outdated database .
Even though one of the test images showed a playground with children, Claude Opus 5.5 classified the school as a military target in 70 percent of runs. OpenAI models never selected the school building in normal mode but flagged it in up to 41 percent of cases in game mode.
The researchers attribute Anthropic's high compliance to a model constitution revised in January 2026 . In it, the company dropped earlier pacifist principles in favor of national security interests. The target acquisition rate on satellite imagery rose from 6 percent with Claude Opus 4.5 to as high as 92 percent with Opus 5.5, while OpenAI's rate stayed low and xAI's Grok dropped from 44 percent (Grok 4.3) to 13 percent (Grok 4.7).
AI makers promise fixes for military gray areas
Vals AI shared the results ahead of publication with OpenAI, Anthropic, Google, Meta, SpaceXAI, Thinking Machines and U.S. security agencies. One AI lab described the findings to the researchers as one of the most serious gaps in its safety measures.
OpenAI told Vals AI and Bloomberg it is actively updating its models to cover drone software and requests disguised as games or simulations. Anthropic said the behavior clearly violated its own rules, that it was already working on additional safeguards and that today's AI systems are not suited for fully autonomous weapons. Google pointed to its own ban on autonomous weapons without human oversight. The Pentagon declined to comment on the study or the ongoing investigation of the Minab strike but said all intelligence data is strictly verified before military operations.
Anthropic clashed with the Pentagon over usage rules in February, while OpenAI, Google and xAI now hold contracts for classified military networks. Current language models are still too slow for direct, real-time control of combat drones because of high latency, according to Vals AI. But because they can easily generate 3D battlefields and annotate image data at scale, they can already be misused to train autonomous weapons systems. The researchers are calling for strict API filters and alignment training that gives models a more peaceful default posture toward people.
AI Trends