Frontier Models Solve Only 15% of Long-Horizon Tasks

# Frontier Models Solve Only 15% of Long-Horizon Tasks

Researchers led by Li et al. published Long-Horizon-Terminal-Bench (LHTB) on July 13, 2026, a new evaluation framework testing how frontier AI models handle real-world agentic work. The benchmark assessed 46 long-horizon terminal tasks across nine categories—including software engineering, scientific computing, and system administration—using dense reward-based grading to award partial credit for intermediate progress.

Top Performer Reaches 28% Task Completion

xAI's Grok 4.5 emerged as the strongest model overall, solving 13 of 46 tasks (28.3%) with an average reward score of 0.51 Arxiv. However, when filtered to high-confidence solves (reward threshold R ≥ 0.95), OpenAI's GPT-5.5 dominated the reliable-completion ranking, resolving 7 of 46 tasks (15.2%) Arxiv. This divergence highlights a critical gap: models can generate partial solutions but struggle to produce consistently correct end-to-end task execution.

Other frontier contenders performed substantially weaker. MiniMax M3 and Kimi K2.7 Code (developed by Moonshot AI) each achieved 6.5% high-confidence solve rates—3 tasks apiece—across the benchmark Arxiv. The low-cost model Hy3 achieved approximately $2.47 per task, while GPT-5.5 cost roughly $21.46 per task and Grok 4.5 about $11 Arxiv.

Implications for Agent Deployment

The results underscore a persistent challenge in the agent economy: production-grade agentic systems require near-perfect task execution, yet frontier models average only 4.3% of runs crossing the R ≥ 0.95 confidence threshold across all 15–18 tested models Arxiv. This gap between raw capability and deployment-ready reliability suggests enterprises relying on agents for software engineering, financial analysis, or system administration face substantial failure risk without extensive guardrails, retrieval-augmented generation, or human oversight.

The benchmark's dense reward methodology—scoring partial credit rather than binary pass/fail—reflects a more nuanced view of agent performance than traditional benchmarks. Yet even with this generosity, the solve rates remain low enough to warrant caution in autonomous high-stakes workflows.

Next Steps

The LHTB dataset is publicly available at zli12321.github.io/LHTB, enabling further research and iterative model improvement. For teams evaluating agent platforms, the benchmark provides concrete, reproducible metrics—replacing vague capability claims with measurable, task-specific outcomes.