Physics agents hit research ceiling despite strong local scores

# Physics agents hit research ceiling despite strong local scores

A new benchmark evaluation published on arXiv reveals that frontier AI models struggle to complete physics research tasks despite showing strong performance on intermediate steps, highlighting a critical gap between agent progress and scientific closure.

Researchers tested seven frontier large language models on 50 expert-verified physics research tasks and found that Claude Opus 5 achieved the highest mean Atomic Score of 6.29/10. However, when measured on complete research closure—the ability to fully resolve a research problem—Claude Opus 5 reached the highest level (RC4) in only 32.7% of attempts.

The disparity points to a fundamental challenge in agentic AI: models can make meaningful progress on individual research steps but fail to synthesize those accomplishments into finished work. The paper's findings indicate that 36.2% of attempts scoring at least 8/10 still failed to reach research closure, suggesting that high local performance does not reliably predict endpoint success.

The closure problem in autonomous research

The evaluation framework, described in "From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research," was published October 15, 2026, with updates through October 16, 2026. The benchmark uses a tiered closure metric (RC1 through RC4) to measure whether agents not only make progress but actually complete defined research objectives.

This distinction matters for real-world deployment. An agent that scores 8/10 on individual physics problem components but fails to deliver a closed solution provides incomplete value to researchers. The gap between local accomplishment and global closure suggests current models may excel at specific subtasks—literature review, hypothesis formulation, calculation—but lack the orchestration or persistence to integrate those steps into publishable results.

Implications for agentic science

The benchmark's methodology—using expert-verified tasks and structured closure levels—provides a more granular picture than typical LLM leaderboards. Rather than measuring general capability, it isolates the specific challenge of research completion, a critical requirement for agents deployed in scientific workflows.

The finding that Claude Opus 5 leads on Atomic Score but still closes research in fewer than one-third of high-scoring attempts suggests the frontier is not raw capability but task completion architecture. Future improvements may require agents with better state management, failure recovery, and endpoint verification rather than raw model scaling alone.

The work joins a growing body of agentic research focused on measurable, domain-specific performance rather than broad benchmarks, establishing concrete evaluation standards for AI agents in expert-driven fields.