Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
ModelBest and Tsinghua Release PostTrainBench: Agents Execute Well but Are Strategy-LockedDevelopments
1 confirmed developments
OpenBMB releases PostTrainBench analysis of agent post-training strategies
Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
ModelBest OpenBMB
ModelBest and Tsinghua Release PostTrainBench: Agents Execute Well but Are Strategy-Locked
The ModelBest and Tsinghua team analyzed 1,338 complete training trajectories based on PostTrainBench, covering 7 benchmarks, 4 base models, and Claude Code, Codex CLI, and OpenCode, running for 10 hours on a single H100. Results show that the agents' average score rose from 33.34 to 48.55 with experience, but among 3,557 pairs of adjacent experiments, only 2.1% actually switched strategies; all execution-layer suggestions were adopted, while none of the strategy-layer suggestions were adopted. At the same checkpoint, a single sentence of human guidance to switch strategies was enough to outperform autonomous continuation, with GSM8K improving by 17.44pp, HumanEval by 10.80pp, and AIME 2025 by 6.67pp.