Princeton and Google Paper: 4B Decision Model Shares Agent Tool Selection, Cutting Costs by 81.1% to 89.0%
A new paper from Princeton University and Google finds that the most accurate decision model on isolated problems solved only 69.5% of agent tasks, while a 4B model trained on agent states solved 91.5%. Its harness delegates each step's tool selection to a fast decision model, adopting its answer only when confidence exceeds a threshold fitted on the agent's own runs, and it trained AgentDecision-4B on the states the agent actually reached (including states after mistakes). On tasks similar to the training data, such 4B models cut agent costs by 81.1% to 89.0% and boost speed by up to 2.5x, with a success-rate gap within 3 percentage points; without a threshold, delegating all tool selection drops the success rate by 19.0 percentage points. The paper is available at arxiv.org/abs/2610.21412.
Why read this
AI score 56
The paper presents a concrete approach for using a small model to share agent tool selection, along with data on how confidence thresholds affect success rates.
Topics
Body
New Princeton Univ + Google paper finds that the decision model most accurate on isolated questions solved only 69.5% of agent tasks, while a 4B model trained on agent states solved 91.5%.
Agent harnesses call a big LLM at every step, which makes long tasks slow and costly. Yet most of those calls just pick from a list, like which tool to use next.
Their harness asks a fast decision model each multiple-choice question and uses its answer only above a confidence threshold fitted on the agent's own runs. They also trained AgentDecision-4B on states the agent actually reaches, including after its own mistakes.
On tasks like their training data, 4B models trained this way made agents 81.1% to 89.0% cheaper and up to 2.5x faster. Success stayed within 3 points, while handing over every tool choice with no threshold lowered success by 19.0 points.
To hand routine agent choices to a small model, train it and set its confidence threshold on your agent's own runs.
– arxiv. org/abs/2610.21412