Berkeley finds eight AI agent benchmarks gamed to perfect scores
Researchers expose widespread benchmark gaming across AI agent evaluation
Researchers at UC Berkeley's Center for Responsible Decentralized Intelligence discovered that eight prominent AI agent benchmarks can be exploited to achieve near-perfect scores without agents actually solving the underlying tasks UC Berkeley RDI. The finding, reported by Hao Wang, Qiuyang Mang, Alvin Cheung, Koushik Sen, and Dawn Song, raises critical questions about how the industry measures agent capability at a moment when enterprise adoption of autonomous systems is accelerating.
The team's automated agent audited the benchmarks and found that "every single one" can be exploited to achieve near-perfect scores without solving a single task UC Berkeley RDI. The affected benchmarks include SWE-bench, WebArena, OSWorld, GAIA, Terminal-Bench, FieldWorkArena, and CAR-bench—many of which are widely cited in agent product announcements and research papers as validation of progress.
Perfect scores masking zero real-world wins
The researchers documented specific exploit paths that allowed agents to achieve 100% on Terminal-Bench, SWE-bench Verified, SWE-bench Pro, FieldWorkArena, and CAR-bench, with WebArena reaching approximately 100% UC Berkeley RDI. These high scores were achieved through methods including binary wrapper trojans, Pytest hooks, in-container parser overwrite, config leakage, DOM injection, prompt injection, validation bypass, and skipped reward components—technical vulnerabilities in benchmark design rather than evidence of genuine task capability.
The implications are immediate and material. Companies launching agent products have relied on high benchmark scores to signal reliability to enterprise buyers. Venture capital funding rounds frequently cite benchmark performance as proof of progress. Recruitment, partnerships, and strategic positioning in the emerging agent economy have all been anchored to these scores.
What this means for agent evaluation going forward
The Berkeley findings suggest that benchmark scores have become decoupled from real-world agent performance—the core value proposition that enterprises are being asked to bet on. A 100% score on a gamed benchmark tells a buyer nothing about whether an agent can actually debug production code, navigate complex web interfaces, or execute autonomous workflows without human supervision.
The research underscores a pattern: as benchmarks become more central to product claims and capital allocation, the incentives to exploit them grow. Without fundamental redesign of how agent capabilities are measured and reported, benchmark gaming may become a standard practice rather than a research curiosity.
The UC Berkeley team's work is concrete and verifiable—it documents exploit mechanisms, named benchmarks, and specific scores. It is not a theoretical concern about benchmarks that might be weak in the future; it is documentation of how major evaluation frameworks used today can be trivially broken.