Coding agents plateau below 45% on realistic benchmarks

# Coding Agents Hit Hard Ceiling on Real-World Code Tasks

Leading AI coding agents have stalled on realistic software engineering problems, solving fewer than 45% of public benchmark tasks and fewer than 20% of proprietary ones, according to performance data now circulating among agent developers and researchers.

The findings, drawn from SWE-Bench Pro leaderboard results, represent a sharp decline from synthetic benchmark performance and underscore a persistent gap between laboratory-grade agent evaluation and production-ready autonomy in code generation and debugging.

The Benchmark Gap

SWE-Bench Pro separates public and proprietary task sets to measure agent performance on increasingly realistic problem conditions. On public tasks—still curated but closer to real-world GitHub issues—top models including Claude Opus 4.8 and GLM-5.2 achieved 69.2% and 62.1% respectively, according to Scale AI's leaderboard. However, a separate evaluation reported that on the same benchmark, leading models OpenAI GPT-5 and Claude Opus 4.1 scored only 23.3% and 23.1%, suggesting significant variance in how results are reported or defined.

The broader consensus from available benchmark pages indicates that across leading coding agents, pass rates on the harder proprietary subset—designed to reflect unreleased, production-grade codebases—remain below 20%, marking a 50%+ drop from public-task performance.

Why It Matters

The plateau matters because SWE-Bench Pro is one of the few evaluation suites explicitly designed to move past synthetic coding tasks toward agent-solvable real problems. Unlike isolated algorithm challenges, SWE-Bench problems require agents to navigate full repositories, read existing tests, modify code in context, and reason about dependencies—the actual work autonomous coding agents must do in enterprise environments.

When proprietary-task performance drops below 20%, it signals that agents still fail on the majority of messy, partially-specified, real-world refactoring and bug-fix work that exists inside actual codebases. This gap has direct consequences for teams evaluating whether to deploy coding agents for production pull requests or repository maintenance.

What's Next

The data suggests that next-generation coding agents will need to improve reasoning over complex context, handle ambiguous specifications, and maintain consistency across large codebases—areas where current models show clear weakness. Benchmark saturation on older suites like SWE-Bench Verified has made newer, harder evaluation sets like SWE-Bench Pro essential for tracking genuine progress in agentic code capability.

Agent teams and researchers continue to iterate on prompt engineering, tool integration, and multi-step reasoning chains to close the gap, but the 20% proprietary ceiling remains a sobering benchmark for the state of production-ready autonomous coding agents.