Browser Use 在 WebVoyager 基准上取得 89.1% 成功率
Browser Use = state of the art Web Agent
Browser Use 在 WebVoyager 基准的 586 个网页任务上取得 89.1% 成功率，并称达到当前最优水平，且完全开源。评测基于 gpt4o，作者对原代码的提示词做了调整并迁移到 Langchain，同时移除了 55 个无法完成或数据过期的任务。作者也指出该数据集主要考察智能体规划能力，并未覆盖 iFrame、Shadow 元素等复杂站点理解场景。
推荐理由
精选AI 评分 64
原文给出 WebVoyager 89.1% 成功率、开源仓库与评测口径说明，读者可据此判断 Web Agent 的当前水平与评测局限。
主题
正文
当前显示原文，中文全文译文尚未就绪。
Engineering
Browser Use achieves state-of-the-art performance on the WebVoyager benchmark with an 89.1% success rate.
Co-founder, CTO
Method; The Results; 🤔 Do these results matter?; Analysis; Manual correction of evaluations; What's Next?; Want to build Web Agents?
We're excited to announce that Browser Use has achieved state-of-the-art performance on the WebVoyager benchmark, with an impressive 89.1% success rate across 586 diverse web tasks. And the best part? We are FULLY open source (repo).
Web Agent Accuracy
Method
We took the existing WebVoyager codebase and slightly changed it according to our needs. The prompts are slightly different. We also migrated pure openai to Langchain. All the code used for testing is available at eval repository.
We ran our evaluation with gpt4o (we will test all other models as well, just give us a little bit of time).
Some tasks are impossible to solve. For Apple doesn't show prices for certain products in the dataset, there are no recipes for chocolate chip cookies etc.
A lot of tasks have dates in the past (kookings, flights), so we just changed the years from 2023 to 2024 or 2024 to 2025 respectively.
We removed 55 tasks. You can find the full list at analysis.ipynb.
The Results
Here's how Browser Use performed across different domains:
Even our "worst" performing domain (Booking.com) still achieved an 80% success rate. Not too shabby!
🤔 Do these results matter?
The WebVoyager dataset is hard. BUT we believe it's not actually testing correct "things". It mostly tests the planning of the agents, but not the actual ability to understand the sites (for example, complex sites with iFrames and Shadow elements are extremely tricky, but not tested with this dataset).
The dataset also includes ambiguous tasks that even humans might interpret differently
Analysis
We believe in transparency, so here are some caveats:
Manual correction of evaluations
The eval model is not good. That's why we added another success criteria - unknown if the eval model is not sure.
Most of the tasks are indeed correct, but some tasks had wrong assesment, and unknown either went into success or failed.
Some tasks were impossible due to outdated data (we removed these); Cloudflare sometimes got grumpy and blocked us; The original evaluation model wasn't great (we manually reviewed edge cases); Some prompts were more ambiguous than your average fortune cookie
What's Next?
We're not stopping here! Coming soon:
make manually labeled items more transparent; add proxy rotations for Cloudflare blocking; test all kinds of models and different setups (Claude, GPT-4o, Llama 3, etc.); test different setups (single vs multiple images, single vs multiple tasks, etc.)
Want to build Web Agents?
Browser Use is 100% Open Source.
Remember: these results were achieved with our base setup - imagine what's possible with all the bells and whistles!
Stay tuned. For more updates join ↓
Published December 15, 2024