DeepSeek V4.1 Flash 1017 Tops the Artificial Analysis Cybersecurity Index Open Model Leaderboard
DeepSeek V4.1 Flash 1017 ranks first among openly available models on the Artificial Analysis Cyber Index with a score of 57%, and second on the overall leaderboard, 12 points behind GPT-6 Sol (69%), which is offered through a trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA, and mainly evaluates agentic cybersecurity defense tasks. Compared with the previous generation DeepSeek V4.1 Flash's 41%, the new version improves by 16 points, mainly from CyberGym-E2E-AA rising from 23% to 76%; its cost per task is $0.42, placing it on the capability-cost Pareto frontier.
Why read this
AI score 58
Artificial Analysis's Cyber Index counts safety refusals toward the score, which helps readers understand part of the reason open-weight models rank highly in cybersecurity evaluations.
Topics
Body
DeepSeek V4.1 Flash 1017 is the new #1 openly-available model in the Artificial Analysis Cyber Index. Other proprietary models may offer better cyber defense performance but are restricted access
@deepseek_ai recently released DeepSeek V4.1 Flash 1017, an update to their flagship open-weights model DeepSeek V4.1 Flash. With an Artificial Analysis Cyber Index score of 57%, DeepSeek V4.1 Flash 1017 is the #1 openly-available model for cyber defense capabilities.
The Artificial Analysis Cyber Index measures capability on agentic cyber defense tasks and is composed of three benchmarks: CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. The Cyber Index primarily evaluates models through their publicly accessible APIs, reflecting the cyber capabilities available to developers.
Many models, particularly proprietary models from OpenAI, Anthropic and Google, may have stronger defensive cyber capabilities but encounter safety refusals that reduce their overall scores. GPT-6 Sol (Daybreak Blue, max), available through OpenAI’s trusted-access program, leads the Index, scoring 32 points above its public counterpart driven by fewer refusals. DeepSeek V4.1 Flash 1017 encounters no safety refusals.
Key results:
➤ DeepSeek V4.1 Flash 1017 has the highest score among publicly-available models on the Artificial Analysis Cyber Index and #2 position overall. DeepSeek V4.1 Flash 1017 (max) scores 57% ahead of the previously leading publicly-available models Grok 4.7 (xhigh) and MiMo-V2.6-Pro by 1 point. It places #2 overall, with the trusted-access model GPT-6 Sol (Daybreak Blue, max) leading by 12 points with a score of 69%
➤ DeepSeek V4.1 Flash 1017 improves on its predecessor DeepSeek V4.1 Flash by 16 points. DeepSeek V4.1 Flash 1017 (max)’s 16-point improvement from 41% to 57% is driven by a substantial 53-point improvement on CyberGym-E2E-AA from 23% to 76%. On CWE-Bench-AA and DeepsecBench-AA it scores 69% and 25%, compared to 68% and 31% for DeepSeek V4.1 Flash (max)
➤ With a Cost per Task of $0.42, DeepSeek V4.1 Flash 1017 sits on the Capability vs. Cost Pareto frontier. DeepSeek V4.1 Flash 1017 costs $0.42 per task, more than 2x MiMo-V2.6-Pro (56%) at $0.18 per task but 4x cheaper than GPT-6 Sol (Daybreak Blue, max, 69%) at $1.77 per task