Understand this story
Replit has released ViBench 2.0, designed to evaluate AI agents' ability to start from an empty project and build a complete product feature by feature. The benchmark includes 17 products, 241 feature requests, and 544 test plans. According to results published on the Replit Blog, GPT-6.1 Sol leads with 65% pass@1, GPT-6 Astra is at 60%, and Opus 5.5 is at 49%, with all three passing over 90% of test plans. Opus 5.5 completes a build in about 70 minutes on average, roughly 2.5 times faster than Sol; DeepSeek V4.1 Flash passes 77% of test plans at about $2.50 per build, but no open-weight model is able to pass a complete build.
AI-generated from public reports
Latest update · Discussion
Amjad Masad joined this discussionDevelopments
1 confirmed developments
Replit releases long-horizon software development benchmark ViBench 2.0
Source:Replit Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 29
Amjad MasadRelated discussion
Amjad Masad joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Zhen LiRelated discussion
Zhen Li joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
ReplitRelated discussion
Replit joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Replit BlogFirst-party
Replit releases ViBench 2.0: a benchmark for long-horizon software building
Replit releases ViBench 2.0 to evaluate AI agents' ability to build a complete product feature by feature starting from an empty project, comprising 17 products, 241 feature requests, and 544 test plans. GPT-6.1 Sol leads with a 65% pass@1, GPT-6 Astra is at 60%, and Opus 5.5 is at 49%, with all three passing over 90% of test plans. Opus 5.5 completes a build in about 70 minutes on average, roughly 2.5 times faster than Sol; DeepSeek V4.1 Flash passes 77% of test plans at about $2.50 per build, but no open-weight model can pass a complete build.
Heat trend for this story
- Current
- 8
- Period peak
- 11 Oct 30 17:00
- 24-hour change
- —
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.