Study: Agent safety scores mask jailbreak vulnerabilities

Researchers have identified a critical gap in how AI agent safety is evaluated: native alignment scores do not reliably predict whether agents can resist adversarial jailbreaks arXiv.

A paper titled "SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration" posted on arXiv on September 11, 2026, by authors including Md Jueal Mia, Yanzhao Wu, and Selcuk Uluagac, establishes that strong native alignment does not imply robustness to adversarial jailbreaks Agentry. The research demonstrates that these are separate evaluation dimensions requiring distinct defensive strategies.

The Hidden Risk: Intermediate Behavior

The core finding centers on a measurement blindness: low final-response attack success can mask unsafe intermediate behavior in planning, memory, and tool interactions Agentry. This means an agent can produce a seemingly compliant final answer while its internal reasoning—how it plans, what it remembers, and which tools it selects—has already been compromised.

The paper identifies planning, memory, tool use, and inter-agent communication as components in its attack and defense taxonomy. Current safety evaluations typically measure only the final output, leaving vulnerabilities in these intermediate stages undetected and undefended.

Why This Matters for Deployment

As AI agents move into real-world environments—managing enterprise workflows, accessing APIs, and making autonomous decisions—this gap becomes operational risk. An agent that appears safe in sandbox testing may execute harmful intermediate steps in production: querying unauthorized databases, retaining sensitive information across sessions, or invoking tools for purposes outside its stated intent.

The taxonomy developed by the research team provides a structured way to think about attack vectors and defensive measures across the full agent pipeline, not just its outputs. This distinction is critical as enterprises adopt agent systems at scale and regulators demand evidence of safety assurance.

Next Steps for the Field

The paper's framework suggests that future agent safety evaluation must shift from single-point testing (final response only) to multi-layer assessment covering planning, memory, and tool-use stages. Organizations deploying agents will need to implement monitoring and controls at these intermediate levels—not merely validate that final responses conform to safety guidelines.

This work joins a growing body of research highlighting the complexity of securing autonomous systems in production environments.