Meta paper finds reinforcement learning post-training improves first-attempt success rate but reduces coverage across multiple attempts
A new Meta paper finds that reinforcement learning post-training makes agents perform better on first attempts, but fewer tasks can be solved within up to 128 attempts. The study tested 14 sets of base models and post-trained models on 3 agent benchmarks, and built the PTGS sampler, which raises temperature for difficult prompts and lowers it for easy prompts. On WebShop, the gemma-4-31B base model solved over 85% of tasks within 128 attempts, while the post-trained version solved 56%; the authors' proposed Sharpening Tax was positive in 36 of 42 model-benchmark combinations, meaning coverage shrank. The authors suggest that when selecting models for retryable agents, one should measure how many tasks the model can ultimately solve, rather than looking only at first-attempt accuracy.
Why read this
AI score 70
The original text uses 42 model-benchmark comparisons to provide quantitative evidence of coverage decline in retryable scenarios, and includes ideas for sampler improvements.
Topics
Body
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attempts.
People assume reinforcement learning teaches agents new skills like tool calling.
That claim comes mostly from math and coding, where base models have already seen huge amounts of data.
They ran 14 base and post-trained model pairs on 3 agentic benchmarks with up to 128 tries per task. They also built PTGS, a sampler that raises temperature on hard prompts and lowers it on easy ones.
On WebShop a gemma-4-31B base model solved over 85% of tasks within 128 tries against 56% for the post-trained version. Their Sharpening Tax came out positive in 36 of 42 model and benchmark pairs, meaning coverage shrank.
Before picking a post-trained model for an agent that can retry, measure how many tasks it ever solves, not just first-try accuracy.