Moonshot weakened Kimi K3.1 Pro's hacking capabilities before open release
Before open release, Moonshot proactively weakened Kimi K3.1 Pro's cyberattack capabilities; the unweakened version had a 90.5% solve rate on its internal exploit task, while the released version was 81%. This result is still far higher than K3's 33.3%, but lower than DeepSeek V4.1-Flash's 85.7%. Moonshot said that exploit skills rise as post-training scale expands, so it removed cyber-related data from mid-training and reinforcement learning, and internal xdev-bench-hard pass@3 dropped from 90.5% to 81.0%.
Why read this
AI score 72
The original text provides the specific scores before and after the weakening and the comparison models, so readers can judge the trade-off regarding cyberattack capabilities at open release.
Topics
View original postMoonshot AI’s first update, the new Kimi K3.1 Pro > Kimi K3.1 Pro scores 65.7% on Terminal-Bench 4.0, 0.7 points behind Claude Opus 5.5 and 7.5 ahead of GPT-6 Astra, at a lower average cost per task than either rival’s top runs. > Moonshot saw exploit skill climb as post-training scaled, so it removed cyber-related data from mid-training and RL; on its internal xdev-bench-hard, pass@3 fell from 90.5% to 81.0%. > The filtered model still solves 81.0% of those exploitation tasks against K3’s 33.3%, which Moonshot reads as general coding gains carrying into cyber work, while a third-party run of GPT-5.6 Sol solved 100%. > On Moonshot’s internal hallucination stress test with tools, false claims fell from 23.4% for K3 to 8.3%, still above GPT-6 Astra’s 5.5% but well below Claude Opus 5.5’s 15.0%. > The biggest jump is on Moonshot’s simulated business-operation test, where scores rose from 8.9 to 37, and K3.1 Pro also tops OfficeQA Pro v2 at 64.4 against Opus 5.5’s 62.2. > Gains are uneven: K3.1 Pro scores 38.1 on Terminal-Bench Science against GPT-6 Astra’s 68.1 and Opus 5.5’s 63.3, while leading Agents’ Last Exam at 43.2 and Humanity’s Last Exam with tools at 70.0. > K3.1 Pro and a Highspeed variant running 2-3x faster are live in Kimi’s apps and API, with full weights due by November 4 for independent reruns.
View original
Moonshot AI’s first update, the new Kimi K3.1 Pro > Kimi K3.1 Pro scores 65.7% on Terminal-Bench 4.0, 0.7 points behind Claude Opus 5.5 and 7.5 ahead of GPT-6 Astra, at a lower average cost per task than either rival’s top runs. > Moonshot saw exploit skill climb as post-training scaled, so it removed cyber-related data from mid-training and RL; on its internal xdev-bench-hard, pass@3 fell from 90.5% to 81.0%. > The filtered model still solves 81.0% of those exploitation tasks against K3’s 33.3%, which Moonshot reads as general coding gains carrying into cyber work, while a third-party run of GPT-5.6 Sol solved 100%. > On Moonshot’s internal hallucination stress test with tools, false claims fell from 23.4% for K3 to 8.3%, still above GPT-6 Astra’s 5.5% but well below Claude Opus 5.5’s 15.0%. > The biggest jump is on Moonshot’s simulated business-operation test, where scores rose from 8.9 to 37, and K3.1 Pro also tops OfficeQA Pro v2 at 64.4 against Opus 5.5’s 62.2. > Gains are uneven: K3.1 Pro scores 38.1 on Terminal-Bench Science against GPT-6 Astra’s 68.1 and Opus 5.5’s 63.3, while leading Agents’ Last Exam at 43.2 and Humanity’s Last Exam with tools at 70.0. > K3.1 Pro and a Highspeed variant running 2-3x faster are live in Kimi’s apps and API, with full weights due by November 4 for independent reruns.
Body
Moonshot deliberately weakened Kimi K3.1 Pro’s hacking ability before the open release.
the uncut version solved 90.5% of in-house exploit tasks, while the released model solved 81%, still well above K3’s 33.3% and behind DeepSeek V4.1-Flash at 85.7% by Moonshot’s count. https://t.co/6HmruAqHXW