Coding agents fail at shared-workspace editing, benchmark finds
Benchmark reveals coding agents stumble when humans edit code
Coding agents remain significantly weaker at handling shared workspaces where users actively modify code, according to a new research evaluation arXiv. The paper, titled "SWE-Touch: Benchmarking Coding Agents When Users Touch the Code," was submitted to the preprint server on August 3, 2026, and tests nine coding models across multiple standard benchmarks to measure performance degradation when human edits interrupt agent workflows.
The core finding: counter-edits by users lowered average resolve rates by 7.7 percentage points on SWE-bench Verified, the gold-standard benchmark for evaluating autonomous code-fixing agents arXiv. The degradation persisted when researchers tested the same scenario on longer-horizon benchmarks including SWE-Bench Pro and DeepSWE, indicating a systemic weakness rather than a single-benchmark artifact.
What the study measured
Authors Yuqiao Tan, Jinxiang Meng, Fangyu Lei, Minzheng Wang, Shizhu He, Jun Zhao, and Kang Liu designed the evaluation to simulate a realistic development pattern: a user begins with a coding task, an agent attempts to resolve it, and the user then edits the workspace mid-task. The research characterizes this as a test of agent awareness in an evolving workspace—a scenario that mirrors collaborative development but one that existing agents handle poorly arXiv.
The 7.7-point drop on SWE-bench Verified represents a material loss of capability. For a benchmark where high-performing models typically achieve resolve rates in the 40–50% range, this magnitude of degradation signals that agents lack robust mechanisms to track, understand, and adapt to code changes initiated outside their own execution loop.
Why this matters for the agent economy
The finding surfaces a gap between lab performance and real-world deployment. Most production coding agents—whether used for bug fixing, feature development, or refactoring—operate in environments where humans remain active contributors. A 7.7-point penalty translates to concrete, measurable drops in task completion and may require human oversight to compensate. Organizations evaluating agents for code automation should interpret published benchmarks with this limitation in mind.
The paper also tested the impact on SWE-Bench Pro and DeepSWE, longer-horizon tasks that more closely approximate the complexity of real-world repositories. The persistence of the degradation across these harder benchmarks suggests the weakness is not a quirk of narrow task design but a fundamental constraint in how current agents maintain state and replan when code context changes mid-execution.
This research joins a growing body of work that stress-tests agent robustness in realistic conditions rather than ideal laboratory setups—a shift that the field increasingly recognizes as essential for understanding deployment readiness.