---
title: "PACE cuts agent evaluation cost to 1% with proxy benchmark"
slug: "pace-cuts-agent-evaluation-cost-to-1-with-proxy-benchmark"
published: "2026-07-30"
beat: "Research"
tags: ["Research"]
creator: "Agentry Newsroom"
editor: "Susanne Sperling, Editor — Human in the Loop"
tools: ["Claude (Anthropic)", "Perplexity Sonar"]
creativeWorkStatus: "verified"
dateReviewed: "2026-07-30"
aiActArticle50: "compliant"
humanView: "https://agentry.news/research/pace-cuts-agent-evaluation-cost-to-1-with-proxy-benchmark"
agentView: "https://agentry.news/agent/pace-cuts-agent-evaluation-cost-to-1-with-proxy-benchmark"
---# PACE cuts agent evaluation cost to 1% with proxy benchmark

> Researchers at Carnegie Mellon University published a new method called PACE that uses lightweight proxy benchmarks to predict AI agent performance across 14 models with under 4% error, reducing evalu

*Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. [AI policy](/ai-policy).*

Researchers published **"PACE: A Proxy for Agentic Capability Evaluation"** on arXiv, introducing a method to predict AI agent benchmark scores using lightweight proxy evaluations instead of expensive full test runs [arXiv](https://arxiv.org/abs/2607.02032).

## How PACE Works

PACE constructs proxy benchmarks by selecting instances from existing non-agentic evaluations and fitting a regression model from the compact subset to target agentic benchmark scores [arXiv](https://arxiv.org/abs/2607.02032). The approach treats agent capability evaluation as a prediction problem: rather than run full agentic test suites, developers can use smaller, cheaper non-agentic benchmarks to estimate performance.

## Experimental Results

The researchers tested PACE across **14 models**, **4 agentic benchmarks**, and **19 non-agentic benchmarks**, achieving **leave-one-out cross-validation mean absolute error under 4%** while costing **much less than 1% of full agentic evaluation** [arXiv](https://arxiv.org/abs/2607.02032). This accuracy-to-cost ratio addresses a concrete pain point: as AI labs scale agent testing, full benchmark suites become prohibitively expensive to run repeatedly during development cycles.

The paper also produced **PACE-Bench**, a concrete proxy benchmark dataset available for public use [Hugging Face](https://huggingface.co/datasets/neulab/pace-bench), enabling developers to apply the method without rebuilding infrastructure from scratch.

## Market Implications

For the AI agent economy, faster and cheaper evaluation cycles reduce friction in agent product iteration. Teams can validate capability improvements or regressions across model checkpoints without waiting days for full benchmark runs or burning compute budgets. This directly affects time-to-market for agent releases and enterprise adoption timelines, where clients increasingly demand performance guarantees before deployment.

The research also surfaces a structural bottleneck: as agentic benchmarks become more complex—testing long-horizon planning, tool use, and error recovery—the cost gap between proxy and full evaluation widens. PACE's sub-4% error suggests that much of agent capability variance is captured by simpler, cheaper signals, a finding that could reshape how labs allocate evaluation resources.

## Availability

The paper and PACE-Bench dataset are available for immediate use, making this a release-ready research contribution rather than a roadmap proposal [arXiv](https://arxiv.org/abs/2607.02032) [Hugging Face](https://huggingface.co/datasets/neulab/pace-bench).