Search “DeepSeek”

Fireworks AI announces evaluation results of DeepSeek V4 Pro on CyberGym

On August 26, 2026, Fireworks AI published evaluation results of DeepSeek V4 Pro 0813 on the CyberGym security benchmark. The test covered 697 common valid tasks; DeepSeek V4 Pro's reward rate was 53.7%, with a cost of $2.502 per successful task, about half that of Kimi K3. In the same batch of results, Kimi K3 had the highest reward rate at 68.4%, with a cost of $4.914 per successful task; GPT-5.5 had a reward rate of 47.6%, with a cost of $9.641 per successful task; Claude Opus 4.8 had a reward rate of 5.9%, with a cost of $33.275 per successful task. Fireworks AI also said that DeepSeek V4 Pro produced native tool calls on all 840 tasks, with no refusals or output truncation; once it passed first-stage validation, its rate of completing the full patch was 94.7%. There are currently no further developments.

View story:Fireworks AI publishes evaluation results of DeepSeek V4 Pro on CyberGym

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

View story:English translation is being prepared.

English translation is being prepared.

View story:English translation is being prepared.

English translation is being prepared.

View story:English translation is being prepared.

Cognition Launches Coding Benchmark FrontierCode 2.0

On October 28, 2026, Cognition launched the coding benchmark FrontierCode 2.0 to evaluate whether the code produced by models can actually be merged; even the highest-scoring Opus 5.5 scored only 45%, showing the benchmark is far from saturated. According to the Cognition blog, compared with version 1.1, the reference solutions modify on average 67% more files and 89% more lines of code, the codebase size is 25% larger, task coverage repositories increased by 55%, and blocking scoring criteria increased by 43%; DeepSeek V4.1 Flash dropped from 43.9% to 19.3%, and Kimi K3 dropped from 44.2% to 17.7%. On the same day, Artificial Analysis released Intelligence Index v5, listing FrontierCode 2.0 as one of the newly added benchmarks, with 60% of evaluations based on private test sets or held-out answers; Claude Opus 5.5 (max) topped the list with 58 points.

blakeandersonw open-source ai-app-playbook

On October 28, 2026, Blake Anderson open-sourced "ai-app-playbook," a set of operating procedures for coding agents to execute, covering product selection, building, publishing, and first-month profitability. The playbook has two layers: a human-facing playbook with chapters 00–13, and an Agent-facing AGENTS.md (Phase 0–14, with each step marked [AGENT] or [HUMAN]); before starting, one must pass the Gate A scoring. The tech stack is fixed as Cloudflare Workers, D1, Better Auth, OpenRouter routing to DeepSeek, Superwall plus Stripe, etc., and explicitly lists items not to use, such as Vercel, Supabase, and Next.js. Currently there is only this open-source release information, with no subsequent developments yet.

PromptArmor says Chinese models have a higher success rate when subjected to cyberattacks

On October 29, 2026, PromptArmor released test results, stating that after testing its most widely circulated attack samples on multiple models, it found that the DeepSeek, Kimi, GLM, and Qwen models were 88% more susceptible to cyberattacks than the GPT, Claude, and Gemini models. The tests covered multiple cyber risk dimensions, including phishing, data exfiltration, and the tendency to run malware, and PromptArmor said the differences were significant. No other institution has yet responded or independently verified the findings, and the conclusion remains solely PromptArmor's own release.

PromptArmor says Chinese models have a higher success rate when subjected to cyberattacks

On October 29, 2026, PromptArmor released test results, stating that after testing its most widely circulated attack samples on multiple models, it found that the DeepSeek, Kimi, GLM, and Qwen models were 88% more susceptible to cyberattacks than the GPT, Claude, and Gemini models. The tests covered multiple cyber risk dimensions, including phishing, data exfiltration, and the tendency to run malware, and PromptArmor said the differences were significant. No other institution has yet responded or independently verified the findings, and the conclusion remains solely PromptArmor's own release.

View story:PromptArmor says Chinese models have a higher success rate when subjected to cyberattacks

PromptArmor says Chinese models are more vulnerable to cyberattacks

On October 29, 2026, PromptArmor released test conclusions stating that DeepSeek, Kimi, GLM, and Qwen models are 88% more likely to be subjected to cyberattacks than GPT, Claude, and Gemini models. The organization said the tests used multiple highly transmissible attack samples, covering cyber risks such as phishing, data exfiltration, and the tendency to run malware. The above conclusions come from PromptArmor's main tweet, which did not explain the specific testing methods or sample size. Currently, no other organization has responded or independently verified the findings, and the status of the incident remains at the stage of a unilateral release by PromptArmor.

PromptArmor says Chinese models are more vulnerable to cyberattacks

On October 29, 2026, PromptArmor released test conclusions stating that DeepSeek, Kimi, GLM, and Qwen models are 88% more likely to be subjected to cyberattacks than GPT, Claude, and Gemini models. The organization said the tests used multiple highly transmissible attack samples, covering cyber risks such as phishing, data exfiltration, and the tendency to run malware. The above conclusions come from PromptArmor's main tweet, which did not explain the specific testing methods or sample size. Currently, no other organization has responded or independently verified the findings, and the status of the incident remains at the stage of a unilateral release by PromptArmor.

View story:PromptArmor says Chinese models are more vulnerable to cyberattacks

Arena ranks Kimi K3.1 Pro (Max) No. 11 in Agent Arena

On October 24, 2026, Arena added Kimi K3.1 Pro (Max) to the Agent Arena rankings, placing it at No. 11. The ranking is based on 4K multi-real-agent sessions, achieving a +7.6% net improvement, up 6 percentage points from Kimi K3 (Max)'s No. 25. Its median cost per task is $1.03, 13% lower than the previous version; it ranks No. 8, No. 9, and No. 10 in the Work, Code, and Chat categories, respectively. Arena said the model's open weights will be released soon, at which point it will become the top-ranked open-source model in Agent Arena, surpassing DeepSeek-v4.1-Flash. There are currently no subsequent developments.

Artificial Analysis says Kimi K3.1 Pro scores 64% on the Cyber Index

Artificial Analysis reports that Kimi K3.1 Pro scored 64% on the Artificial Analysis Cyber Index, ranking first among openly available models. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA; K3.1 Pro scored 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its cost per task is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42. Earlier the same day, Artificial Analysis ranked DeepSeek V4.1 Flash 1017 first among openly available models with a score of 57%, second on the overall leaderboard, 12 points behind GPT-6 Sol (69%), which is offered through a trusted access program; this version improved by 16 points over its predecessor DeepSeek V4.1 Flash's 41%, mainly due to CyberGym-E2E-AA rising from 23% to 76%.

Artificial Analysis evaluates MiniMax-M3.1-Flash with a score of 42

Artificial Analysis released its evaluation on October 19, 2026; MiniMax-M3.1-Flash scored 42 on its Intelligence Index, 13 points higher than MiniMax-M3, at less than half the latter's cost per task. The same evaluation shows the model scored 48% on Humanity's Last Exam and 22% on CritPt, both higher than GLM-5.3-Flash and Gemini 3.8 Flash, and slightly lower than DeepSeek V4.1 Flash 1017. The model has a 1M-token context window, supports text and image inputs, is priced at $0.15/$0.60 per 1M input/output tokens, and its weights are currently closed-source; MiniMax announced that it will open the weights on October 23.

Aleph Alpha releases the Kolibri-1 open-weight model

On October 16, 2026, Aleph Alpha released the open-weight model Kolibri-1. According to heise online, the model has 78B total parameters and uses a MoE architecture with 384 experts and 6 routed experts, activating only 3.46B parameters during inference, with a context of 256K tokens that can be extrapolated to about 1 million tokens. The model focuses on German and English, and its tokenizer segments German words more token-efficiently; for example, the German Bundestag glossary requires only 1,137 tokens, whereas the compared DeepSeek-V4.1-Flash requires 1,910 tokens. Pretraining used 20 trillion tokens, with data up to June 18, 2026. The report's author tested it and said its answers were correct and its German reasoning process evident, and said the Merlin-Arthur protocol makes the model hallucinate less in benchmarks, but its actual performance still needs to be tested in real-world use. There are currently no subsequent developments.

Artificial Analysis Launches Dual HealthBench Leaderboards

Artificial Analysis has launched leaderboards for OpenAI's two health dialogue benchmarks, HealthBench and HealthBench Professional. HealthBench contains 5,000 conversations with general users and health workers, while HealthBench Professional consists of 525 conversations all aimed at clinicians. The leaderboard results show that on HealthBench Professional, GPT-6 Astra (max) ranks first with a length-adjusted score of 66.0%, followed by GPT-6.1 Sol (max) at 65.1% and Claude Sonnet 5.5 (max with fallback) at 63.0%; on HealthBench, Claude Sonnet 5.5 (max with fallback) leads with 62.1%, while DeepSeek V4.1 Flash (max) scores 60.9%. In addition, Claude Opus 5.5 and Claude Fable 5.1 give average answers of about 5,900 characters, losing more than 11 points after length adjustment. There are currently no further developments.

Teacher shares practice of a 9-agent collective lesson-preparation pipeline

On October 17, 2026, a high school English teacher, under the identity "Ku Sir," posted on the Tencent Cloud Developer Community, sharing how he and a 3-person lesson-preparation group transformed collective lesson preparation into a four-stage pipeline of 9 AI agents plus 3 human roles. The article states that the lesson-preparation loop that originally spanned two weeks was run through in a single morning in a test in early September; four sets of prompts respectively handled three-dimensional attribution of student learning conditions, four-part discussion opinions each no more than 100 characters, an 80-minute debate hosting script, and three-dimensional scoring and archiving, and the consensus execution rate changed from frequently left unfinished to 100%. The author also recorded that a 6-point drop in the second quality-check score was actually due to a change in the scoring rubric, teachers' resistance to format constraints, and a member on a business trip being covered by the deputy preparer filling three gaps, and stated that the SOP is general across disciplines and that the prompts can run on Doubao, Kimi, ChatGPT, and DeepSeek. At present, this practice is still presented only as the author's self-account, with no subsequent independent progress.