Meta 论文发现强化学习后训练提升首次成功率却降低多次尝试覆盖率
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attem
Meta 新论文发现，强化学习后训练让智能体首次尝试表现更好，但在最多 128 次尝试中能找到解的任务反而更少。研究在 3 个智能体基准上测试了 14 组基础模型与后训练模型，并构建了 PTGS 采样器，对困难提示提高温度、对简单提示降低温度。在 WebShop 上，gemma-4-31B 基础模型在 128 次尝试内解决超过 85% 的任务，后训练版本为 56%；作者提出的 Sharpening Tax 在 42 组模型与基准组合中有 36 组为正，意味着覆盖率缩小。作者建议为可重试的智能体选型时，应衡量模型最终能解多少任务，而不只看首次准确率。
推荐理由
AI 评分 70
原文用 42 组模型与基准对比给出可重试场景下覆盖率下降的量化证据，并附采样器改进思路。
主题
正文
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attempts.
People assume reinforcement learning teaches agents new skills like tool calling.
That claim comes mostly from math and coding, where base models have already seen huge amounts of data.
They ran 14 base and post-trained model pairs on 3 agentic benchmarks with up to 128 tries per task. They also built PTGS, a sampler that raises temperature on hard prompts and lowers it on easy ones.
On WebShop a gemma-4-31B base model solved over 85% of tasks within 128 tries against 56% for the post-trained version. Their Sharpening Tax came out positive in 36 of 42 model and benchmark pairs, meaning coverage shrank.
Before picking a post-trained model for an agent that can retry, measure how many tasks it ever solves, not just first-try accuracy.