测试自动化提示注入攻击在智能体环境中的效果，发现黑盒方法更优且攻击依赖模型能力。
Assessing Automated Prompt Injection Attacks in Agentic Environments
- 用黑盒优化（TAP）和白盒方法（GCG）评估智能体场景下的攻击效果。
- 黑盒方法性能远超梯度法，因白盒方法在合理算力下不稳定。
- 攻击效果取决于攻击模型能力，安全微调模型会拒绝生成恶意提示。
间接提示注入对与不可信外部数据交互的LLM智能体构成严重威胁，但针对真实智能体环境的自动化攻击方法研究仍不足。本文在AgentDojo框架下，将白盒（GCG）和黑盒（TAP）方法适配至智能体场景，评估了跨越四个领域、80个任务对及多种模型的攻击表现。结果表明，黑盒优化显著优于基于梯度的方法，这归因于GCG在合理计算预算下的优化不稳定性。此外，TAP的效果依赖于攻击模型本身：更强的模型产生更有效的注入，而经过安全微调的模型会拒绝生成对抗性提示。任务通用型攻击可在未见任务和分布外领域有效迁移，但在较小开源模型上优化的攻击无法成功转移到前沿模型（如GPT-5）。这些发现表明，自动化提示注入是可信但模型依赖的威胁，实现模型无关攻击仍面临重大障碍。
原文摘要 · Abstract (English)
Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a comprehensive empirical evaluation of automated prompt injection attacks against LLM agents, adapting both white-box (GCG) and black-box (TAP) methods to the agentic setting within the AgentDojo framework. We evaluate across 80 task pairs spanning four domains and multiple models, and find that black-box optimization substantially outperforms gradient-based methods, a gap we attribute to GCG's optimization instability under reasonable compute budgets. We also find that TAP's effectiveness depends on the attacker model, as both general capability and safety tuning affect attack success--stronger models produce more effective injections, while safety-tuned attackers can refuse to generate adversarial prompts. Task-universal attacks transfer effectively to unseen tasks and out-of-distribution domains, but attacks optimized on smaller open-source models do not transfer to frontier models like GPT-5. These findings highlight automated prompt injection as a credible but model-dependent threat, with significant barriers remaining for model-agnostic exploitation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。