揭示大模型代理内存中毒攻击机制与防御盲区
From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents
- 识别4种内存写入路径与9类系统漏洞
- 构建6类攻击分类与评估基准MPBench
- 现有提示注入防护对内存攻击无效
记忆是人工智能代理的核心组件，使其能在交互中积累知识并提升性能。然而持久化记忆引入了内存中毒风险——单次恶意记忆写入即可长期影响代理行为。本文系统研究基于大语言模型的代理中的内存中毒攻击，识别出4种内存写入通道及模型能力、系统提示设计和代理架构中的9类结构性漏洞，使这些通道可被利用。基于此，我们提出六类内存中毒攻击的分类体系，并设计了评估基准MPBench。实验表明，越倾向于频繁写入和读取记忆的代理，越易被攻击。此外，现有提示注入防御措施无法覆盖内存中毒攻击。研究为理解并缓解针对智能代理的内存中毒攻击提供了基础。
原文摘要 · Abstract (English)
Memory is a core component of AI agents, enabling them to accumulate knowledge across interactions and improve performance. However, persistent memory introduces the risk of memory poisoning, where a single adversarial memory write can exert long-term influence over agent behavior. We present a systematic study of memory poisoning in LLM-based agents. We identify four memory write channels and nine structural vulnerabilities in model capabilities, system prompt design, and agent system architecture that make these channels exploitable. Based on these vulnerabilities, we develop a taxonomy of six classes of memory poisoning attacks. Furthermore, we design MPBench -- a benchmark for evaluating memory poisoning attacks, and show that agents designed to write and retrieve memory more aggressively are more exploitable. We also show that existing prompt injection defenses fail to cover memory poisoning attacks. Our findings provide a foundation for understanding and mitigating memory poisoning attacks against AI agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。