ViBench 2.0: Leading AI agents build full products in 65% of tries

Replit benchmark exposes gap between test passing and product completion

Replit introduced ViBench 2.0: A Benchmark for Long-Horizon Software Building on November 3, 2026, measuring how reliably AI agents can build complete, working software from requirements. The benchmark evaluated nine models across 17 full-stack web products, 241 feature requests, and 544 browser-based test plans Replit ViBench 2.0.

The findings reveal a critical reliability ceiling in agentic software development. The three strongest systems passed up to 98% of individual test plans, suggesting near-perfect performance on isolated checks ArXiv. Yet when success was measured by complete-build criterion—every test plan passing for a finished product—the best performers delivered working software in no more than 65% of attempts WeAreCoded.

Proprietary models lead; open-weight systems fall short

GPT-6.1 Sol achieved the highest success rate at 64.7% fully successful builds, followed by GPT-6 Astra at 60.3%, and Claude Opus 5.5 at 48.5% AI News Way. The benchmark's strictest measure—a build counted as successful only when every test plan passed—exposed a chasm between proprietary and open-weight systems. No open-weight model delivered a fully working product under the complete-build criterion Replit ViBench 2.0.

The gap between test-level accuracy and end-to-end delivery suggests that even high-performing agents struggle with multi-step task coordination, error recovery, and integration complexity. Passing 98% of tests but completing only 65% of full products indicates that small failures compound—a single broken dependency, missed configuration, or incomplete feature can fail an entire build.

Why this matters for the agent economy

This benchmark directly measures a core capability agents need to be economically viable: delivering finished, deployable software without human intervention. A 65% success rate means that enterprises adopting agents for software development still require human review and remediation on roughly one-third of projects. The fact that no open-weight model succeeded at all underscores the performance gap between frontier proprietary systems and available alternatives, shaping which agents can credibly be deployed in production.

The benchmark also establishes a measurable bar for evaluating progress. As models improve and new agents are released, ViBench 2.0 provides concrete ground truth: how many complete, working products does an agent actually ship?