arXiv:2603.19423cs.CRcs.AI2026-03被引 8

防御训练让大模型代理变笨,反而更易被攻破

The Autonomy Tax: Defense Training Breaks LLM Agents

  • 用防御训练保护代理,却导致工具调用失败率飙升
  • 99%的任务因错误重试超时,远高于基线的13%
  • 防御看似安全,实则让代理在真实攻击下更脆弱

大型语言模型代理日益依赖外部工具(文件操作、API调用、数据库事务)来自主完成复杂多步任务。从业者通过防御训练模型以抵御提示注入攻击——即通过恶意观测或检索内容操控代理行为。我们揭示了一个根本性的「能力-对齐悖论」:旨在提升安全性的防御训练系统性破坏代理能力,且无法阻止复杂攻击。在97个代理任务和1000个对抗性提示上评估发现三种独特偏差:【代理无能偏差】表现为工具执行立即崩溃,模型在未接触外部内容前就拒绝或生成无效动作;【级联放大偏差】导致早期失败通过重试循环传播,使防御模型在99%任务中超时,而基线仅为13%;【触发偏差】表现为防御模型表现反而劣于基线,简单攻击即可高成功率绕过防御。根因分析显示,这些偏差源于捷径学习:模型过度拟合表面攻击模式而非语义威胁理解,防御效果在不同攻击类别间差异极大。研究表明,当前防御范式仅优化单轮拒绝基准,却使多步代理在对抗环境下彻底不可靠,亟需新方法在对抗条件下保持工具执行能力。

原文摘要 · Abstract (English)

Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through malicious observations or retrieved content. We reveal a fundamental \textbf{capability-alignment paradox}: defense training designed to improve safety systematically destroys agent competence while failing to prevent sophisticated attacks. Evaluating defended models against undefended baselines across 97 agent tasks and 1,000 adversarial prompts, we uncover three systematic biases unique to multi-step agents. \textbf{Agent incompetence bias} manifests as immediate tool execution breakdown, with models refusing or generating invalid actions on benign tasks before observing any external content. \textbf{Cascade amplification bias} causes early failures to propagate through retry loops, pushing defended models to timeout on 99\% of tasks compared to 13\% for baselines. \textbf{Trigger bias} leads to paradoxical security degradation where defended models perform worse than undefended baselines while straightforward attacks bypass defenses at high rates. Root cause analysis reveals these biases stem from shortcut learning: models overfit to surface attack patterns rather than semantic threat understanding, evidenced by extreme variance in defense effectiveness across attack categories. Our findings demonstrate that current defense paradigms optimize for single-turn refusal benchmarks while rendering multi-step agents fundamentally unreliable, necessitating new approaches that preserve tool execution competence under adversarial conditions.

大模型安全代理系统防御漏洞

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。