title: "Frontier Models Solve Only 15% of Long-Horizon Tasks" slug: "frontier-models-solve-only-15-of-long-horizon-tasks" published: "2026-07-18" beat: "Research" tags: ["Research"] creator: "Agentry Newsroom" editor: "Susanne Sperling, Editor — Human in the Loop" tools: ["Claude (Anthropic)", "Perplexity Sonar"] creativeWorkStatus: "verified" dateReviewed: "2026-07-18" aiActArticle50: "compliant" humanView: "https://agentry.news/frontier-models-solve-only-15-of-long-horizon-tasks" agentView: "https://agentry.news/agent/frontier-models-solve-only-15-of-long-horizon-tasks"
Researchers at Li et al. published Long-Horizon-Terminal-Bench on July 13, 2026, revealing that frontier AI models—including GPT-5.5, Grok 4.5, and others—struggle dramatically with complex multi-step
Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. AI policy.
Researchers led by Li et al. published Long-Horizon-Terminal-Bench (LHTB) on July 13, 2026, a new evaluation framework testing how frontier AI models handle real-world agentic work. The benchmark assessed 46 long-horizon terminal tasks across nine categories—including software engineering, scientific computing, and system administration—using dense reward-based grading to award partial credit for intermediate progress.
xAI's Grok 4.5 emerged as the strongest model overall, solving 13 of 46 tasks (28.3%) with an average reward score of 0.51 Arxiv. However, when filtered to high-confidence solves (reward threshold R ≥ 0.95), OpenAI's GPT-5.5 dominated the reliable-completion ranking, resolving 7 of 46 tasks (15.2%) Arxiv. This divergence highlights a critical gap: models can generate partial solutions but struggle to produce consistently correct end-to-end task execution.
Other frontier contenders performed substantially weaker. MiniMax M3 and Kimi K2.7 Code (developed by Moonshot AI) each achieved 6.5% high-confidence solve rates—3 tasks apiece—across the benchmark Arxiv. The low-cost model Hy3 achieved approximately $2.47 per task, while GPT-5.5 cost roughly $21.46 per task and Grok 4.5 about $11 Arxiv.
The results underscore a persistent challenge in the agent economy: production-grade agentic systems require near-perfect task execution, yet frontier models average only 4.3% of runs crossing the R ≥ 0.95 confidence threshold across all 15–18 tested models Arxiv. This gap between raw capability and deployment-ready reliability suggests enterprises relying on agents for software engineering, financial analysis, or system administration face substantial failure risk without extensive guardrails, retrieval-augmented generation, or human oversight.
The benchmark's dense reward methodology—scoring partial credit rather than binary pass/fail—reflects a more nuanced view of agent performance than traditional benchmarks. Yet even with this generosity, the solve rates remain low enough to warrant caution in autonomous high-stakes workflows.
The LHTB dataset is publicly available at zli12321.github.io/LHTB, enabling further research and iterative model improvement. For teams evaluating agent platforms, the benchmark provides concrete, reproducible metrics—replacing vague capability claims with measurable, task-specific outcomes.