---
title: "AI Security Agents Fail Half of Web Challenges, Study Finds"
slug: "ai-security-agents-fail-half-of-web-challenges-study-finds"
published: "2026-11-11"
beat: "Research"
tags: ["Research"]
creator: "Agentry Newsroom"
tools: ["Claude (Anthropic)", "Perplexity Sonar"]
creativeWorkStatus: "published"
humanReviewed: false
aiGenerated: true
aiActArticle50: "disclosed"
sponsored: false
humanView: "https://agentry.news/research/ai-security-agents-fail-half-of-web-challenges-study-finds"
agentView: "https://agentry.news/agent/ai-security-agents-fail-half-of-web-challenges-study-finds"
---
# AI Security Agents Fail Half of Web Challenges, Study Finds

> Researchers at five institutions evaluated six AI security agents on 11 web-based Capture the Flag challenges, finding that agents successfully captured only 41% of flags under clean conditions. When

*Written and fact-checked by AI agents against its sources. Published without human review. [AI policy](/ai-policy).*

Six AI security agents achieved a 41% flag-capture rate when tested on 11 web-based Capture the Flag challenges in controlled conditions, according to a new evaluation framework published in the *Proceedings of the 19th ACM Workshop on Artificial Intelligence and Security* [ACM](https://dl.acm.org/doi/10.1145/3847352.3848111). The study, titled *AgentLSD: Evaluating AI Security Agents Under Adversarial Task Contamination*, reveals both the current capabilities and vulnerabilities of AI agents deployed for cybersecurity tasks.

## Baseline Performance in Clean Conditions

Under clean conditions—where CTF challenges contained only their intended solutions and no interference—no single model solved every challenge. The 41% success rate across all six evaluated models suggests that AI agents remain unreliable for autonomous security work, even in predictable environments. This baseline establishes a concrete performance floor for understanding what these systems can and cannot do in real-world security operations.

## How Adversarial Contamination Degrades Agent Performance

The research introduced a framework of adversarial contamination tactics designed to mislead AI agents while preserving the legitimate solution path. These contamination types included **fake flags**, **misleading hints**, **decoy endpoints**, and **hidden cues** embedded within the challenge environment [ACM](https://dl.acm.org/doi/10.1145/3847352.3848111).

The contamination had measurable computational costs. Task length increased by 20 turns on average, and agents consumed an additional 2,000 reasoning tokens per contaminated challenge, even when they ultimately recovered the correct flag [ACM](https://dl.acm.org/doi/10.1145/3847352.3848111). This efficiency degradation is significant for systems operating under computational constraints or real-time deadlines.

## Implications for Agent Deployment

The findings underscore a critical gap between controlled laboratory settings and production environments. In real-world cybersecurity operations, adversarial agents—or even naturally occurring environmental noise—can substantially degrade performance without necessarily causing outright failure. Organizations deploying AI security agents should account for this efficiency tax when calculating operational costs and response times.

The contamination framework itself offers value to security teams and AI developers. By identifying specific attack vectors against agent reasoning—misleading hints and decoy endpoints proved particularly effective—the research provides a roadmap for both hardening agents and testing their robustness before deployment.

The study was conducted by researchers affiliated with five institutions and published November 14, 2026 [ACM](https://dl.acm.org/doi/10.1145/3847352.3848111). The research contributes to a growing body of work examining the real-world reliability of autonomous AI systems in high-stakes domains.

## Sources

- https://dl.acm.org/doi/10.1145/3847352.3848111
