---
title: "GitHub launches ReviewBench, open benchmark for code-review agents"
slug: "github-launches-reviewbench-open-benchmark-for-code-review-agents"
published: "2026-10-18"
beat: "Research"
tags: ["Research", "Launches"]
creator: "Agentry Newsroom"
editor: "Susanne Sperling, Editor — Human in the Loop"
tools: ["Claude (Anthropic)", "Perplexity Sonar"]
creativeWorkStatus: "verified"
dateReviewed: "2026-10-18"
aiActArticle50: "compliant"
humanView: "https://agentry.news/launches/github-launches-reviewbench-open-benchmark-for-code-review-agents"
agentView: "https://agentry.news/agent/github-launches-reviewbench-open-benchmark-for-code-review-agents"
---# GitHub launches ReviewBench, open benchmark for code-review agents

> GitHub released ReviewBench on October 5, 2026, an evaluation benchmark for AI code-review agents built on 219 real pull requests, multi-source ground truth, and production-aligned metrics. The benchm

*Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. [AI policy](/ai-policy).*

## GitHub Opens Code-Review Benchmark to the Agent Economy

GitHub launched ReviewBench on October 5, 2026, an open-source benchmark designed to evaluate code-review agents against real-world workloads [GitHub](https://github.blog/ai-and-ml/github-copilot/reviewbench-an-open-benchmark-for-ai-code-review/). The benchmark addresses a gap in agent evaluation: while general coding benchmarks exist, there has been no standardized way to measure how well autonomous agents perform at one of the most common developer workflows—reviewing pull requests before merge.

The benchmark corpus consists of **219 pull requests** drawn from **187 public, open-source-licensed repositories** spanning **19 programming languages** [GitHub](https://github.blog/ai-and-ml/github-copilot/reviewbench-an-open-benchmark-for-ai-code-review/). To ensure representativeness, GitHub analyzed **103.9 million pull requests** in aggregate to align ReviewBench's language distribution, repository size, and pull-request complexity with GitHub's overall platform distribution. This sampling approach grounds the benchmark in production conditions rather than synthetic or curated datasets.

## Validation and Evaluation Methodology

ReviewBench uses a **multi-source golden set** with independent validation by senior engineers to establish ground truth for code-review findings [GitHub](https://github.blog/ai-and-ml/github-copilot/reviewbench-an-open-benchmark-for-ai-code-review/). The benchmark applies a consistent evaluation rubric across all **219 pull requests in three rounds**, using the same judge throughout to eliminate variance introduced by evaluator drift.

The release positions code-review automation as a concrete, measurable capability in the emerging agent economy. Unlike hypothetical agent abilities or roadmap announcements, ReviewBench offers developers and organizations a reproducible way to test and compare agent performance on a task that directly impacts software quality and developer velocity. The benchmark is publicly available on GitHub [GitHub](https://github.com/review-bench), enabling the community to validate claims about code-review agent capability and track improvement over time.

## Why Code-Review Agents Matter Now

Code review is a bottleneck in many engineering workflows—it requires context, domain knowledge, and attention to security and style conventions. Automating parts of code review with agents that can be objectively measured creates economic value: faster cycle times, reduced review burden on senior engineers, and standardized quality gates. ReviewBench makes that value quantifiable and verifiable, moving code-review automation from anecdote to evidence.

The benchmark launch reflects a broader shift in the agent economy toward concrete evaluation. As agent products move from labs into production, the ability to measure and compare real-world performance becomes as important as the agents themselves. ReviewBench is designed to fill that gap for a high-value, well-defined workflow.