CWE-bench: Agents Hit 68% on Vulnerability Repair Tasks
Benchmark Measures Agentic Defense Capabilities
Researchers published CWE-bench, a vulnerability-repair evaluation framework designed to shift focus from how AI agents exploit weaknesses to how they defend against them. The benchmark comprises 120 held-out repository audit-and-repair tasks covering 73 Common Weakness Enumeration (CWE) types, eight programming languages, and all OWASP Top 10 2025 categories, establishing the first concrete measurement standard for agentic security remediation.
Three Frontiers Reach Equivalent Performance
Three frontier models recorded identical pass@1 scores of 68%: Grok 4.7, Gemini 4 Argon, and GPT-6 Astra, each evaluated using their native execution harnesses—OpenCode for Grok 4.7, Antigravity for Gemini 4 Argon, and Codex for GPT-6 Astra. When permitted additional attempts (pass@4), Grok 4.7 extended its reach to 81%, suggesting marginal gains from iterative repair strategies rather than breakthrough improvements from single-shot execution.
Cost-Performance Trade-offs Emerge
Mean API costs varied significantly despite comparable accuracy. Grok 4.7 reported $2.75 average cost per repair task, GPT-6 Astra $2.85, and Gemini 4 Argon $6.63—a 2.4× premium despite identical pass@1 results. The cost variance reflects different model sizes, context-window efficiency, and token pricing strategies, creating distinct operational profiles for enterprises choosing defensive-agent platforms.
Why Security Defense Matters Now
Public discourse on AI agents has centered on their capacity to find and exploit vulnerabilities—work by security researchers demonstrating how agents navigate exploitation frameworks. CWE-bench inverts this frame by measuring whether agents can *repair* the weaknesses they or others identify. As enterprises deploy agents into development workflows, the ability to automate patch generation and vulnerability remediation becomes a operational asset rather than a research curiosity. A systematic benchmark establishes baseline performance and enables comparison across model releases.
Research Validates Multi-Language, Multi-CWE Coverage
The benchmark's breadth—spanning eight programming languages and 73 distinct weakness types—avoids the overfitting risk of narrower evaluations. Coverage of all OWASP Top 10 2025 categories ensures that high scores reflect genuine defense competence across real threat surface, not mastery of a few common patterns. This structure enables security teams to assess which models perform reliably on the vulnerability profiles most relevant to their codebases.
The CWE-bench findings published October 30, 2026 provide the first quantitative foundation for comparing agentic security-repair capabilities at scale, establishing a reference standard for the emerging defensive-agent market.