---
title: "LoopsBench: Coding agents solve just 25% of loop tasks"
slug: "loopsbench-coding-agents-solve-just-25-of-loop-tasks"
published: "2026-08-18"
beat: "Research"
tags: ["Research"]
creator: "Agentry Newsroom"
editor: "Susanne Sperling, Editor — Human in the Loop"
tools: ["Claude (Anthropic)", "Perplexity Sonar"]
creativeWorkStatus: "verified"
dateReviewed: "2026-08-18"
aiActArticle50: "compliant"
humanView: "https://agentry.news/research/loopsbench-coding-agents-solve-just-25-of-loop-tasks"
agentView: "https://agentry.news/agent/loopsbench-coding-agents-solve-just-25-of-loop-tasks"
---# LoopsBench: Coding agents solve just 25% of loop tasks

> Researchers at University of Wisconsin–Madison released LoopsBench on July 31, 2026, a benchmark for long-horizon loop engineering in AI coding agents. The strongest configuration achieved only a 25.0

*Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. [AI policy](/ai-policy).*

Researchers Frederic Sala and Aws Albarghouthi released LoopsBench on July 31, 2026, introducing a benchmark designed to measure how well AI coding agents handle loop engineering—the iterative, multi-step problem-solving patterns common in real software development [arXiv](https://arxiv.org/pdf/2608.00267). The paper, titled **"LOOPSBENCH: From Harness Engineering to Loop Engineering in Benchmarking Coding Agent,"** evaluated agent performance on long-horizon coding tasks and found that the strongest configuration achieved only a **25.00% solve rate** [arXiv](https://arxiv.org/pdf/2608.00267).

## The Gap in Loop Engineering

LoopsBench marks a shift in how researchers evaluate coding agents. Rather than treating agent evaluation as a static, single-pass measurement, the benchmark focuses on loop engineering—the ability of agents to iterate, debug, and refine solutions across multiple rounds. This reflects real-world software development, where coders frequently revisit and improve their work.

The 25% benchmark result underscores a persistent challenge in the agent ecosystem: even the best-performing configurations struggle with sustained, iterative problem-solving. This gap has become visible across recent coding-agent evaluations, where agents that perform well on single-turn tasks frequently fail when required to maintain context, correct errors, and adapt strategies over multiple steps [Codex Daniel Vaughan](https://codex.danielvaughan.com/2026/08/05/loopsbench-loop-engineering-benchmark-dag-regression-obligations-codex-cli-goal-mode-long-horizon/).

## Implications for Agent Deployment

The LoopsBench findings carry direct implications for enterprises deploying autonomous coding agents. Tasks requiring iteration—refactoring, regression fixing, and multi-file refactoring—represent a large share of real engineering work. A 25% solve rate on such tasks suggests that current agent architectures are not yet reliable for unsupervised, long-horizon coding jobs without human oversight.

The research also highlights why benchmarking methodology matters. Earlier benchmarks like SWE-Bench Pro have shown coding agents below 45% on real-world tasks [Agentry News](https://agentry.news/agent/coding-agents-stall-below-45-on-swe-bench-pro-real-tasks). LoopsBench adds specificity: it isolates the loop-engineering component of the problem, making it possible to diagnose where agent performance breaks down.

## Concrete Benchmark Impact

The paper's arXiv identifier is **arXiv:2608.00267**, and the full preprint is available for peer review [arXiv](https://arxiv.org/pdf/2608.00267). LoopsBench is accompanied by open infrastructure—the paper references tooling and harnesses available on GitHub for reproducible evaluation [GitHub](https://github.com/AMAP-ML/LongHorizon-Harness).

This benchmark contributes to a growing body of concrete evidence about agent capability ceilings. Unlike roadmap announcements or capability claims, LoopsBench provides a measurable standard against which future agent generations can be tested. Vendors and researchers building coding agents now have a tool to quantify whether architectural or training improvements actually translate to better performance on sustained, multi-step tasks.