GitHub launches ReviewBench, open benchmark for code-review agents

GitHub Opens Code-Review Benchmark to the Agent Economy

GitHub launched ReviewBench on October 5, 2026, an open-source benchmark designed to evaluate code-review agents against real-world workloads GitHub. The benchmark addresses a gap in agent evaluation: while general coding benchmarks exist, there has been no standardized way to measure how well autonomous agents perform at one of the most common developer workflows—reviewing pull requests before merge.

The benchmark corpus consists of 219 pull requests drawn from 187 public, open-source-licensed repositories spanning 19 programming languages GitHub. To ensure representativeness, GitHub analyzed 103.9 million pull requests in aggregate to align ReviewBench's language distribution, repository size, and pull-request complexity with GitHub's overall platform distribution. This sampling approach grounds the benchmark in production conditions rather than synthetic or curated datasets.

Validation and Evaluation Methodology

ReviewBench uses a multi-source golden set with independent validation by senior engineers to establish ground truth for code-review findings GitHub. The benchmark applies a consistent evaluation rubric across all 219 pull requests in three rounds, using the same judge throughout to eliminate variance introduced by evaluator drift.

The release positions code-review automation as a concrete, measurable capability in the emerging agent economy. Unlike hypothetical agent abilities or roadmap announcements, ReviewBench offers developers and organizations a reproducible way to test and compare agent performance on a task that directly impacts software quality and developer velocity. The benchmark is publicly available on GitHub GitHub, enabling the community to validate claims about code-review agent capability and track improvement over time.

Why Code-Review Agents Matter Now

Code review is a bottleneck in many engineering workflows—it requires context, domain knowledge, and attention to security and style conventions. Automating parts of code review with agents that can be objectively measured creates economic value: faster cycle times, reduced review burden on senior engineers, and standardized quality gates. ReviewBench makes that value quantifiable and verifiable, moving code-review automation from anecdote to evidence.

The benchmark launch reflects a broader shift in the agent economy toward concrete evaluation. As agent products move from labs into production, the ability to measure and compare real-world performance becomes as important as the agents themselves. ReviewBench is designed to fill that gap for a high-value, well-defined workflow.