Browser Use 发布 BU Bench V1：100 项浏览器任务评测基准
Browser Agent Benchmark: Comparing LLM Models for Web Automation
Browser Use 开源了首个评测基准 BU Bench V1，包含 100 项从五个来源挑选的高难度浏览器任务，任务集加密以防训练污染。评测由 gemini-2.5-flash 以真/假二元判定打分，与人工标注的一致率为 87%。结果显示 Browser Use Cloud 得分 78%，比最好的开源模型高 16 个百分点，且以约每小时 14 个任务成为最快方案；基准代码已开源于 github.com/browser-use/benchmark。
正文
当前显示原文，中文全文译文尚未就绪。
Engineering
We benchmark every major LLM on 100 hard browser tasks. Browser Use Cloud scores 78%, 16 points ahead of the best open-source model.
Evaluations
100 hard browser tasks, one leaderboard; The judge; Results; Why Cloud scores higher; Using the benchmark
100 hard browser tasks, one leaderboard
To truly understand our agent performance, we built a suite of internal tools for evaluating our agent in a standardized and repeatable way so we can compare versions and models and continuously improve. We take evaluations seriously. As of now, we have over 600,000 tasks run in testing.
This is our first open source benchmark. BU Bench V1: 100 hand-selected tasks that are hard but possible, drawn from five established sources.
Every task was run many times with different LLMs, agent settings, and frameworks. Too-easy tasks were removed. Tasks majority-voted impossible and never completed were removed. What's left is hard and verified completable.
The task set is encrypted to prevent LLM training contamination.
The judge
Real websites can't be judged deterministically. We use an LLM judge (gemini-2.5-flash) with a simple true/false verdict. Rubric-based scoring sounds better in theory, but in practice LLMs give middling scores to both successes and failures. Binary verdicts are more reliable.
We hand-labeled 200 traces and measured alignment. The judge agrees with human judgments 87% of the time, differing only on partial successes and technicalities.
To ensure consistency across models, the same judge LLM, prompt, and inputs are used for every evaluation.
Results
Browser Use Cloud leads at 78%, 16 points ahead of the best open-source model. Each model was evaluated multiple times and results include error bars (standard error).
Update (June 2026): Anthropic's claude-fable-5 set a new high score of 80.0% running on the open-source library, beating the next-best frontier model (GPT 5.5) by 12 points — though at $580.87 in API cost per 100-task run. See the full breakdown in our Claude Fable benchmark post.
The "Open Source" column means running the open-source Browser Use library with that LLM. "OSS + Cloud LLM" means the open-source library using our ChatBrowserUse-2 model, which is specifically optimized for browser automation. "Cloud" is the fully managed Browser Use Cloud agent.
The throughput plot tells the rest of the story. Browser Use Cloud (bu-ultra) is both the most accurate and the fastest at ~14 tasks per hour. GPT-5 is the slowest at ~6 tasks per hour. Each bu-ultra step is slower than a smaller LLM, but it completes tasks in far fewer steps, so total wall-clock time is lower. Speed matters in production.
Why Cloud scores higher
Browser Use Cloud is not just a model. It combines a purpose-built agent with our own browser infrastructure: stealth proxies, CAPTCHA solving, persistent filesystem, and optimized tool orchestration. The 16-point gap over the best open-source model comes from this full-stack optimization, not just a better LLM.
For users who need custom tools or self-hosting, the open-source library with ChatBrowserUse-2 (63.3%) still outperforms every standalone open-source model.
Using the benchmark
The benchmark is open source at github.com/browser-use/benchmark. Clone the repo, set your API keys, and run uv run python run_eval.py.
A single run of 100 tasks takes ~3 hours and costs ~10onthebasicBrowserUseplan.Moreexpensivemodelslikeclaude−opus−4−6cancostnearly100 in LLM API calls per run.
We want LLM providers to use this benchmark to test and improve their models on real agentic browsing tasks. If you need to run evaluations at larger scale, contact support@browser-use.com.
Try Browser Use Cloud
Published January 31, 2026