title: "SWE-Bench ProMax: Coding Agents Hit 41.2% on Multilingual Refactor" slug: "swe-bench-promax-coding-agents-hit-412-on-multilingual-refactor" published: "2026-08-19" beat: "Research" tags: ["Research"] creator: "Agentry Newsroom" editor: "Susanne Sperling, Editor — Human in the Loop" tools: ["Claude (Anthropic)", "Perplexity Sonar"] creativeWorkStatus: "verified" dateReviewed: "2026-08-19" aiActArticle50: "compliant" humanView: "https://agentry.news/research/swe-bench-promax-coding-agents-hit-412-on-multilingual-refactor" agentView: "https://agentry.news/agent/swe-bench-promax-coding-agents-hit-412-on-multilingual-refactor"
Researchers published SWE-Bench ProMax, a new benchmark for software engineering agents, on August 10, 2026 at COLM 2026. The study found that the best-performing setup—GPT-5.2 under the OpenHands sca
Drafted by an AI agent. Verified by Susanne Sperling, Editor — Human in the Loop. AI policy.
Researchers have published a new evaluation of how well current AI agents perform on complex software engineering tasks. The paper, SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring, appeared on arXiv on August 10, 2026 as a conference paper at COLM 2026.
The benchmark extends prior work on agent evaluation by testing agents on multilingual code refactoring at scale. According to the paper, the best-performing setup achieved a 41.2% resolve rate, with GPT-5.2 running under the OpenHands scaffold identified as the top result.
The benchmark targets a concrete capability: whether coding agents can successfully resolve real refactoring tasks across multiple programming languages. Rather than testing toy problems or synthetic tasks, SWE-Bench ProMax uses instances drawn from actual software engineering workflows, making the 41.2% ceiling a meaningful signal of current agent limitations.
The OpenHands scaffold—an agentic framework that structures how agents interact with code repositories and tools—proved the most effective environment for the best frontier model. However, the relatively modest resolve rate underscores that even frontier-grade models struggle with the complexity and nuance of production refactoring work.
This result arrives as companies and teams increasingly deploy coding agents into enterprise workflows. A 41.2% resolve rate on multilingual refactoring means that human review and intervention remain essential; agents cannot yet be treated as fully autonomous on complex code tasks. The benchmark provides a concrete measurement point for vendors and researchers tracking progress in this domain.
The multilingual aspect of SWE-Bench ProMax is significant for global adoption. Code refactoring rarely happens in a single language—real projects mix Python, JavaScript, Go, Rust, and others. By testing across languages, the benchmark reflects real-world complexity that earlier, single-language evaluations did not capture.
The paper is now available on arXiv, Hugging Face Papers, and via community discussions on X. The work provides a baseline for future agent improvements and a public benchmark that teams can use to measure their own models and scaffolds against frontier performance.