Understand this story
The Anthropic team proposes Agent4Sci, a long-horizon automated hill-climbing evaluation framework for scientific machine learning tasks, covering 11 scientific domains, 20 benchmarks, and 25 tasks. According to a 36Kr·AI report, in experiments the average relative improvements of Claude Opus 4.6, Claude Sonnet 4.6, Gemini 3.1 Pro, and Codex (GPT-5.4) were 10.4%, 9.6%, 6.8%, and 4.4%, respectively, and all 20 benchmarks were improved by at least one agent. Trajectory analysis shows that the improvements mainly came from routine machine learning engineering techniques such as learning rate adjustment and backbone network replacement; after adding three plugin skills, Opus's average relative improvement rose to 20.1%. This work was accepted to the NeurIPS 2026 Evaluations and Datasets track. There are currently no further developments.
AI-generated from public reports
Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
36Kr · AI
Anthropic proposes the Agent4Sci evaluation framework: AI scientific research improvements mostly come from engineering tricks
The Anthropic team proposes Agent4Sci, a long-horizon automated hill-climbing evaluation framework for scientific machine learning tasks, covering 11 scientific fields, 20 benchmarks, and 25 tasks. In the experiments, the average relative improvements of Claude Opus 4.6, Claude Sonnet 4.6, Gemini 3.1 Pro, and Codex (GPT-5.4) were 10.4%, 9.6%, 6.8%, and 4.4%, respectively, and all 20 benchmarks were improved by at least one Agent. Trajectory analysis shows that the improvements mainly come from conventional machine learning engineering tricks such as learning rate tuning and backbone replacement; after adding three plugin skills, Opus's average relative improvement rose to 20.1%. This work was accepted to the NeurIPS 2026 Evaluations and Datasets track.