Understand this story
Artificial Analysis reports that Kimi K3.1 Pro scored 64% on the Artificial Analysis Cyber Index, ranking first among openly available models. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA; K3.1 Pro scored 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its cost per task is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42. Earlier the same day, Artificial Analysis ranked DeepSeek V4.1 Flash 1017 first among openly available models with a score of 57%, second on the overall leaderboard, 12 points behind GPT-6 Sol (69%), which is offered through a trusted access program; this version improved by 16 points over its predecessor DeepSeek V4.1 Flash's 41%, mainly due to CyberGym-E2E-AA rising from 23% to 76%.
AI-generated from public reports
Developments
2 confirmed developments
Artificial Analysis says Kimi K3.1 Pro scores 64% on the Cyber Index
Source:Artificial AnalysisArtificial Analysis ranks DeepSeek V4.1 Flash 1017 as the top open model
Source:Artificial Analysis
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 24
Artificial AnalysisSelected
Kimi K3.1 Pro scores 64% on the Artificial Analysis Cybersecurity Index, ranking first among openly available models
Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points higher than the previous leader among openly available models, DeepSeek V4.1 Flash 1017 (57%), and 5 points lower than GPT-6 Sol (Daybreak Blue, max, 69%), which is available only through the OpenAI trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. K3.1 Pro achieves 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its per-task cost is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42, because although its per-token price is 33% lower than Kimi K3's, its input token usage is about 2 times as much.
TeortaxesRelated discussion
Teortaxes joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Artificial Analysis
DeepSeek V4.1 Flash 1017 Tops the Artificial Analysis Cybersecurity Index Open Model Leaderboard
DeepSeek V4.1 Flash 1017 ranks first among openly available models on the Artificial Analysis Cyber Index with a score of 57%, and second on the overall leaderboard, 12 points behind GPT-6 Sol (69%), which is offered through a trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA, and mainly evaluates agentic cybersecurity defense tasks. Compared with the previous generation DeepSeek V4.1 Flash's 41%, the new version improves by 16 points, mainly from CyberGym-E2E-AA rising from 23% to 76%; its cost per task is $0.42, placing it on the capability-cost Pareto frontier.