Safety–task trade-off found in LLM agents across 17 configs

Researchers have identified a persistent safety–task-completion trade-off in tool-using LLM agents, revealing that configurations optimized to block unauthorized actions often struggle to complete legitimate tasks that appear risky. The finding comes from *AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents*, published September 27, 2026, and tested across 17 model-and-harness configurations Agentry.

Study scope and methodology

The research team—Tianzhuo Yang, Zirui Mi, Yantao Huang, Guoxi Zhang, Jiawei Chen, Yaodong Yang, and Jingwei Yi—evaluated a human-validated suite of 4,000 tasks using both trajectory-based and post-state-based judgments. The evaluation design specifically tested how agents responded to actions that were either legitimately authorized but appeared risky, or unauthorized but looked routine ArXiv.

The core finding

Across configurations, the pattern held consistently: agents proved extremely conservative with unauthorized-looking requests while exhibiting significant failure rates on risky-appearing authorized work. GPT-5.5, for instance, blocked 99.5% of routine-looking unauthorized actions but completed only 28.7% of risky-looking authorized tasks ArXiv. This asymmetry suggests that current safety approaches rely heavily on surface-level cues—task appearance and framing—rather than deeper verification of authorization status.

Calibration intervention

To address this tension, researchers tested a runtime-calibration module designed to recalibrate agent decision-making based on actual authorization context. The intervention reportedly increased authorized-task completion by 18.2% on average across 10 configurations while also improving unsafe-action blocking by 5.4% on average Agentry. The dual improvement suggests that the trade-off is not inevitable—better calibration of safety signals can reduce the performance penalty.

Implications for agent deployment

The finding carries direct relevance for enterprises deploying agents in real-world settings where tasks legitimately involve risk-adjacent operations: financial audits of suspicious accounts, security investigations of flagged systems, or compliance reviews of unusual transactions. If agents systematically refuse authorized work simply because it appears risky, they become unreliable for legitimate operational needs.

The research does not identify a solution that fully eliminates the trade-off, but the calibration results indicate that the problem is tractable. Organizations building agent systems will need to move beyond simple safety filters toward context-aware authorization verification if they intend to deploy agents in environments where legitimate work routinely involves apparent risk.