arXiv:2606.26479cs.CRcs.AI2026-06

提出新评估方法,验证大模型代理防御提示注入的实效性

Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents

论文配图:Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents
图 1 · 摘自论文原文
  • 将外部防御机制归类为完整性保护、引用监控等经典安全模型
  • 在动态攻击下,防护成功率从25.8%降至4.2%,未被手写攻击突破
  • 适用于研究安全防御的学者和实践者,尤其关注真实攻击场景

近期工作(2024至2026年)趋向于通过外部确定性策略而非模型内拒绝指令来防御工具型大模型代理的间接提示注入。如CaMeL、FIDES、Progent、RTBAS、FORGE等系统利用能力控制、信息流标签和参考监视实现该策略,已在AgentDojo基准上报告近乎消除攻击。本文贡献两点:一是将这些外部防御归纳为经典的完整性保护(Biba)、引用监控与最小权限模型,建立结构化比较框架;二是指出所有现有验证均基于静态基准(固定注入尝试),而此类方法曾因适应性攻击失效——我们提出适应性评估的威胁模型与协议,并独立复现扩展Progent的分析,在使用Qwen2.5-7B(单H200自托管)的条件下进行测试。三次运行平均显示,防御使攻击成功率从25.8%降至4.2%,手写自适应攻击未使其回升(2.6%)。此结果为弱模型上的小样本数据点,白盒优化攻击(如GCG)仍待验证。结果支持但不确证‘确定性外部防护更难被自适应攻击攻破’的假设。

原文摘要 · Abstract (English)

Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a deterministic policy that mediates the agent's actions. Systems such as CaMeL, FIDES, Progent, RTBAS, and FORGE realize this with capabilities, information-flow labels, and reference monitors, and several report near-elimination of attacks on the AgentDojo benchmark. We make two contributions. First, we organize these out-of-band defenses as instances of classical integrity protection (Biba), reference monitoring, and least privilege, yielding a structured comparison of what they do and do not cover. Second, we warn that every one of them is validated only on static benchmarks (a fixed set of injection attempts), the same methodology that made in-band defenses look strong until adaptive, defense-aware attacks broke twelve of them at over 90% success; we specify the threat model and protocol an adaptive evaluation requires. We then run that protocol as an independent reproduction and extension of Progent's own adaptive-attack analysis, on AgentDojo, with an open-weight agent (Qwen2.5-7B) self-hosted on a single H200, a setting its authors did not test. Averaged over three runs, the defense held: Progent cut mean attack success roughly sixfold (25.8% to 4.2%), and a hand-crafted adaptive attack did not raise it (2.6%). This is one small-scale data point on a weak model with a single black-box attack template; a stronger optimized (white-box GCG) attack remains open. The result is consistent with, but does not establish, the hypothesis that deterministic out-of-band enforcement is a harder target for an adaptive attacker than in-band detection.

大模型安全提示注入防御评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。