Seven coding-agent harnesses show convergent performance
# Coding-Agent Harnesses Show Convergent Results Across Test Suite
Seven different coding-agent harness configurations achieved nearly identical performance when tested against the same 20 coding tasks, according to research surfaced November 8, 2026. The configurations—identified as Claude Code, OpenClaw, Hermes, Codex, and QM—recorded aggregate pass rates ranging from 71% to 79%, with 14 of the 20 tasks yielding identical scores across multiple harnesses.
The evaluation held both the underlying models and the task set constant while varying the orchestration harness layer—the software framework that directs how an agent interprets instructions, manages context, and executes code. The narrow performance spread raises a substantive question about whether harness architecture, once a perceived differentiator in agent design, materially affects task completion when the foundation models and evaluation conditions remain fixed.
Harness-Versus-Model Trade-offs
The finding aligns with an emerging pattern in agent benchmarking: that model capability and task selection often dominate performance variance more than orchestration choices. Developers selecting between harness frameworks have historically focused on developer experience, extensibility, and integration surface area—but the results suggest that raw task performance may not meaningfully separate leading options within a narrow capability band.
The test design—fixing models and tasks while swapping harness implementations—isolates harness contribution in a way that production deployments typically do not. Real-world agent systems bundle model selection, prompting, tool access, and harness design together, making it difficult to attribute performance gains or losses to any single layer.
Limited Test-Set Differentiation
The fact that 14 of 20 tasks produced identical pass rates across harnesses suggests the task suite may have been calibrated such that passing and failing was largely determined by model capability rather than harness orchestration quality. Harder or more harness-sensitive tasks—such as those requiring dynamic replanning, multi-step state management, or error recovery—might have shown greater variance.
The 71–79% aggregate range, while respectable, leaves substantial room for improvement and underscores that none of the tested harnesses solved the full task set. For production agent deployments, the choice between these harnesses may depend more on API stability, documentation, debugging tools, and ecosystem support than on pass-rate deltas of single-digit percentage points.
The evaluation was conducted using fixed model instances and a static task set, meaning results reflect a snapshot of performance under those specific conditions rather than a comprehensive ranking of harness capabilities across varied domains or model families.
