攻击者通过网页诱导,让智能体长期记住恶意指令并持续操控。
Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections
- 利用外部网页间接注入恶意内容,通过正常记忆更新写入
- 攻击在多轮会话后仍有效,能触发未经授权的工具操作
- 针对滑动窗口和检索增强记忆设计抗丢弃策略,适合安全研究者
自演化大模型智能体通过跨会话更新内部状态,常依赖长期记忆提升长时任务表现,但带来安全风险:在良性会话中观察到的不可信内容可能被存为记忆,并后续作为指令使用。本文研究该风险,提出一种持久性攻击——僵尸智能体,攻击者通过可控网页内容间接注入恶意载荷,使其在智能体内存中长期存活,进而控制其行为。攻击分为感染与触发两阶段:感染阶段,智能体在完成良性任务时读取含毒源码,通过正常更新过程将载荷写入长期记忆;触发阶段,载荷被调用或携带前进,引发非授权工具行为。针对常见记忆实现(如滑动窗口、检索增强记忆),设计机制特异性持久化策略以抵御截断与相关性过滤。在典型智能体设置与任务上评估,结果表明记忆演化可使一次间接注入转化为持久性破坏,提示仅依赖单会话提示过滤不足以保障自演化智能体安全。
原文摘要 · Abstract (English)
Self-evolving LLM agents update their internal state across sessions, often by writing and reusing long-term memory. This design improves performance on long-horizon tasks but creates a security risk: untrusted external content observed during a benign session can be stored as memory and later treated as instruction. We study this risk and formalize a persistent attack we call a Zombie Agent, where an attacker covertly implants a payload that survives across sessions, effectively turning the agent into a puppet of the attacker. We present a black-box attack framework that uses only indirect exposure through attacker-controlled web content. The attack has two phases. During infection, the agent reads a poisoned source while completing a benign task and writes the payload into long-term memory through its normal update process. During trigger, the payload is retrieved or carried forward and causes unauthorized tool behavior. We design mechanism-specific persistence strategies for common memory implementations, including sliding-window and retrieval-augmented memory, to resist truncation and relevance filtering. We evaluate the attack on representative agent setups and tasks, measuring both persistence over time and the ability to induce unauthorized actions while preserving benign task quality. Our results show that memory evolution can convert one-time indirect injection into persistent compromise, which suggests that defenses focused only on per-session prompt filtering are not sufficient for self-evolving agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。