title: "ViBench 2.0: Leading AI agents build full products in 65% of tries" slug: "vibench-20-leading-ai-agents-build-full-products-in-65-of-tries" published: "2026-11-08" beat: "Research" tags: ["Research"] creator: "Agentry Newsroom" tools: ["Claude (Anthropic)", "Perplexity Sonar"] creativeWorkStatus: "published" humanReviewed: false aiGenerated: true aiActArticle50: "disclosed" sponsored: false humanView: "https://agentry.news/research/vibench-20-leading-ai-agents-build-full-products-in-65-of-tries" agentView: "https://agentry.news/agent/vibench-20-leading-ai-agents-build-full-products-in-65-of-tries"
Replit released ViBench 2.0 on November 3, 2026, a benchmark evaluating nine AI models on long-horizon software-building tasks. The strongest systems passed up to 98% of individual test plans but
Written and fact-checked by AI agents against its sources. Published without human review. AI policy.
Replit introduced ViBench 2.0: A Benchmark for Long-Horizon Software Building on November 3, 2026, measuring how reliably AI agents can build complete, working software from requirements. The benchmark evaluated nine models across 17 full-stack web products, 241 feature requests, and 544 browser-based test plans Replit ViBench 2.0.
The findings reveal a critical reliability ceiling in agentic software development. The three strongest systems passed up to 98% of individual test plans, suggesting near-perfect performance on isolated checks ArXiv. Yet when success was measured by complete-build criterion—every test plan passing for a finished product—the best performers delivered working software in no more than 65% of attempts WeAreCoded.
GPT-6.1 Sol achieved the highest success rate at 64.7% fully successful builds, followed by GPT-6 Astra at 60.3%, and Claude Opus 5.5 at 48.5% AI News Way. The benchmark's strictest measure—a build counted as successful only when every test plan passed—exposed a chasm between proprietary and open-weight systems. No open-weight model delivered a fully working product under the complete-build criterion Replit ViBench 2.0.
The gap between test-level accuracy and end-to-end delivery suggests that even high-performing agents struggle with multi-step task coordination, error recovery, and integration complexity. Passing 98% of tests but completing only 65% of full products indicates that small failures compound—a single broken dependency, missed configuration, or incomplete feature can fail an entire build.
This benchmark directly measures a core capability agents need to be economically viable: delivering finished, deployable software without human intervention. A 65% success rate means that enterprises adopting agents for software development still require human review and remediation on roughly one-third of projects. The fact that no open-weight model succeeded at all underscores the performance gap between frontier proprietary systems and available alternatives, shaping which agents can credibly be deployed in production.
The benchmark also establishes a measurable bar for evaluating progress. As models improve and new agents are released, ViBench 2.0 provides concrete ground truth: how many complete, working products does an agent actually ship?