通过多轮检查修复恶意指令，提升大模型代理的安全性
Soft Instruction De-escalation Defense
- 采用多轮迭代净化输入，逐次检测并修正潜在恶意指令
- 在对抗测试中将攻击成功率降至85%以下（原100%）
- 适合高风险场景的智能代理系统，如自动决策与工具调用
大型语言模型（LLMs）越来越多地部署于与外部环境交互的智能体系统中，这使其在处理不可信数据时易受提示注入攻击。为应对这一问题，我们提出SIC（Soft Instruction Control）——一种适用于工具增强型LLM智能体的简单而有效的迭代提示净化循环。该方法反复检查输入数据中可能影响智能体行为的指令内容；若发现此类内容，则将其重写、遮蔽或移除，并重新评估结果。该过程持续进行，直至输入被清理或达到最大迭代次数；若仍存在命令式指令类内容，智能体将停止运行以确保安全。通过允许多轮处理，本方法承认单次重写可能失败，但可在此后步骤中捕捉并纠正遗漏的注入。尽管如此，最坏情况分析表明，强攻击者仍可通过嵌入非命令式工作流实现15%的攻击成功率（ASR），但整体防御门槛显著提高。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an external environment; this makes them susceptible to prompt injections when dealing with untrusted data. To overcome this limitation, we propose SIC (Soft Instruction Control)-a simple yet effective iterative prompt sanitization loop designed for tool-augmented LLM agents. Our method repeatedly inspects incoming data for instructions that could compromise agent behavior. If such content is found, the malicious content is rewritten, masked, or removed, and the result is re-evaluated. The process continues until the input is clean or a maximum iteration limit is reached; if imperative instruction-like content remains, the agent halts to ensure security. By allowing multiple passes, our approach acknowledges that individual rewrites may fail but enables the system to catch and correct missed injections in later steps. Although immediately useful, worst-case analysis shows that SIC is not infallible; strong adversary can still get a 15% ASR by embedding non-imperative workflows. This nonetheless raises the bar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。