通过解析工具返回结果,有效防御大模型间接提示注入攻击。
Defense Against Indirect Prompt Injection via Tool Result Parsing
- 解析工具输出数据,精准过滤恶意指令。
- 攻击成功率(ASR)创最低纪录,实用性能优异。
- 无需额外训练,适合部署在实时系统中。
随着大模型代理从数字助手转向自主系统与机器人中的物理控制器,间接提示注入攻击风险日益加剧。攻击者可通过在工具调用结果中嵌入恶意指令,劫持代理决策流程执行未授权操作。现有防御方法主要分为两类:一类是训练专用检测模型,但存在计算开销大、需频繁更新的问题;另一类是基于提示工程的防御,虽灵活但多数方法攻击成功率(ASR)仍较高,鲁棒性不足。本文提出一种新方法,通过精确解析工具返回结果,在保留有效数据的同时有效过滤注入的恶意代码。实验表明,该方法在保持高实用性(UA)的同时,达到当前最低的攻击成功率(ASR),显著优于现有方案。代码已开源于 GitHub。
原文摘要 · Abstract (English)
As LLM agents transition from digital assistants to physical controllers in autonomous systems and robotics, they face an escalating threat from indirect prompt injection. By embedding adversarial instructions into the results of tool calls, attackers can hijack the agent's decision-making process to execute unauthorized actions. This vulnerability poses a significant risk as agents gain more direct control over physical environments. Existing defense mechanisms against Indirect Prompt Injection (IPI) generally fall into two categories. The first involves training dedicated detection models; however, this approach entails high computational overhead for both training and inference, and requires frequent updates to keep pace with evolving attack vectors. Alternatively, prompt-based methods leverage the inherent capabilities of LLMs to detect or ignore malicious instructions via prompt engineering. Despite their flexibility, most current prompt-based defenses suffer from high Attack Success Rates (ASR), demonstrating limited robustness against sophisticated injection attacks. In this paper, we propose a novel method that provides LLMs with precise data via tool result parsing while effectively filtering out injected malicious code. Our approach achieves competitive Utility under Attack (UA) while maintaining the lowest Attack Success Rate (ASR) to date, significantly outperforming existing methods. Code is available at GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。