title: "AI agents can design evals—but still botch the details" slug: "ai-agents-can-design-evalsbut-still-botch-the-details" published: "2026-10-20" beat: "Research" tags: ["Research"] creator: "Agentry Newsroom" editor: "Susanne Sperling, Editor — Human in the Loop" tools: ["Claude (Anthropic)", "Perplexity Sonar"] creativeWorkStatus: "verified" dateReviewed: "2026-10-20" aiActArticle50: "compliant" humanView: "https://agentry.news/research/ai-agents-can-design-evalsbut-still-botch-the-details" agentView: "https://agentry.news/agent/ai-agents-can-design-evalsbut-still-botch-the-details"
Researchers at UC San Diego and Carnegie Mellon tested whether AI agents could autonomously create evaluation benchmarks, finding that nine agent systems successfully repaired tasks through pilot feed
Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. AI policy.
Researchers at UC San Diego and Carnegie Mellon have demonstrated that AI agents can autonomously design executable evaluation benchmarks, but their self-created evaluators still contain structural flaws that undermine reliability arXiv.
The team introduced EvalResearchBench, a framework that tested nine different researcher agents tasked with selecting or synthesizing evaluation tasks, implementing graders, and revising them through pilot testing—all under fixed time and API budgets arXiv. The benchmark compared the resulting evaluators against 14 established targets spanning coding, collaborative work, and reasoning domains.
Agent-designed evaluators demonstrated meaningful capability. When given pilot feedback from test runs, the agents successfully identified and repaired broken tasks and graders, showing adaptive improvement. This signals that autonomous evaluation design is not theoretical—agents can move beyond static task selection to iterative refinement.
The nine researcher agents—evaluated across 13 candidate models—produced evaluators that showed concrete alignment with human-curated benchmarks, suggesting the core premise is viable: autonomous systems can participate in the benchmark design pipeline.
But autonomy came with costs. Despite repairs through iteration, agent-designed evaluators still exhibited three categories of failure:
Truncation errors: Some evaluators cut off answer submissions prematurely, blocking valid responses from being scored.
Budget exhaustion: Agents allocated evaluation resources unevenly, depleting token or call budgets partway through a domain, forcing incomplete scoring.
Weighting imbalance: A small number of questions often came to dominate overall domain scores, skewing results and failing to measure breadth.
These problems persisted even after pilot-testing cycles, indicating that agent-driven evaluation design requires human oversight to catch systemic allocations failures that individual task repair cannot solve arXiv.
Evaluation is a critical agent action: it determines what capabilities are measured, how performance is ranked, and which systems advance to production. If agents design their own benchmarks without catching structural flaws, the resulting evaluations become unreliable signals—potentially favoring systems that exploit weighting imbalances or budget constraints rather than genuine capability.
The work suggests a near-term role for human-agent collaboration in benchmark creation. Agents can handle task synthesis, grading logic, and iterative revision; humans must validate resource allocation, test coverage, and structural integrity before evaluators ship.
The research team—Yaolun Zhang, Tianyi Xu, Yujie Zhao, Jishen Zhao, Qingyun Wu, and Huazheng Wang—submitted the full benchmark to arXiv on October 3, 2026, with updates through October 6, 2026.