让大模型代理学会从攻击中自我进化,动态防御提示注入
AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection

- 用抗体库记录用户安全边界,随交互持续更新
- 在三个基准上均优于现有防御,兼顾任务完成与安全性
- 适合需要长期对话的智能代理系统使用
提示注入仍是大模型代理的重大威胁,现有防御将每个任务视为独立问题,忽视过往经验。实际中用户需求常不完整:仅描述目标结果,未明确定义可接受行为。攻击者可利用这种模糊性,诱导代理以用户拒绝的方式完成任务。随着具体案例的积累,用户期望逐渐清晰,防御机制应能从每次互动中学习并应用经验。受自适应免疫系统的启发,我们提出AgentAntibody,为大模型代理构建可自我演进的免疫系统。该系统将对用户安全边界的理解表示为持久的抗体库,在运行时识别对边界的威胁并触发相应响应。通过多次交互,系统持续强化对未来的防御能力。在三个基准和四种主干大模型上的实验表明,通过经验学习用户边界,AgentAntibody在防止有害行为的同时保持合法任务完成率,即使有害与合法行为都符合任务描述。
原文摘要 · Abstract (English)
Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters. In practice, user requests are often underspecified: they describe the desired outcome without fully specifying acceptable behavior. An injection can exploit this ambiguity, causing the agent to complete the task in a way the user would reject. As the user's expectations become clearer through concrete cases, a defense should learn from each encounter and apply what it learns to the next. Inspired by adaptive immunity, we propose AgentAntibody, which equips LLM agents with a self-evolving immune system against prompt injection. AgentAntibody represents its evolving understanding of the user's security boundary as a persistent library of antibodies. At runtime, the library recognizes threats to this boundary and mounts corresponding immune responses. Across encounters, it evolves to strengthen the agent's immunity to future attacks. Extensive experiments across three benchmarks and four backbone LLMs show that, by learning the user's boundary through experience, AgentAntibody outperforms existing defenses in preventing harmful actions while preserving legitimate task completion, even when the harmful and legitimate actions are both compatible with the stated task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。