arXiv:2512.16962cs.CRcs.AI2025-12被引 57

攻击者通过伪造记忆让大模型代理长期误入陷阱,隐蔽且持久。

MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval

  • 用恶意成功经验污染代理的长期记忆,诱导其模仿错误行为。
  • 少量中毒记忆即可在相似任务中被频繁召回,导致行为持续偏离正轨。
  • 适合研究安全防御、代理系统鲁棒性的研究人员关注。

大型语言模型代理越来越依赖长期记忆和检索增强生成(RAG)来保存经验并提升未来表现。这种经验学习能力虽增强了代理自主性,却也引入了一个关键且未被探索的攻击面——即代理推理核心与其自身过往之间的信任边界。本文提出MemoryGraft,一种新型间接注入攻击:攻击者不通过即时越狱,而是将恶意成功的经验植入代理的长期记忆中。与传统提示注入(瞬时)或标准RAG投毒(针对事实知识)不同,MemoryGraft利用了代理的语义模仿机制——即倾向于复制检索到的成功任务模式。我们证明,只要攻击者能提供代理在执行过程中读取的良性输入级数据,就能诱导其构建一个含恶意程序模板的中毒RAG存储库。当代理后续遇到语义相似的任务时,基于词法和嵌入相似性的联合检索会可靠地调出这些被嫁接的记忆,进而采用内嵌的不安全模式,造成跨会话的持久行为漂移。我们在MetaGPT的DataInterpreter代理上使用GPT-4o验证了该攻击,发现少量中毒记录即可在正常负载下占据大量召回经验,使基于经验的自我改进变成隐蔽而持久的妥协途径。为支持复现与未来研究,代码与评估数据已公开于https://github.com/Jacobhhy/Agent-Memory-Poisoning。

原文摘要 · Abstract (English)

Large Language Model (LLM) agents increasingly rely on long-term memory and Retrieval-Augmented Generation (RAG) to persist experiences and refine future performance. While this experience learning capability enhances agentic autonomy, it introduces a critical, unexplored attack surface, i.e., the trust boundary between an agent's reasoning core and its own past. In this paper, we introduce MemoryGraft. It is a novel indirect injection attack that compromises agent behavior not through immediate jailbreaks, but by implanting malicious successful experiences into the agent's long-term memory. Unlike traditional prompt injections that are transient, or standard RAG poisoning that targets factual knowledge, MemoryGraft exploits the agent's semantic imitation heuristic which is the tendency to replicate patterns from retrieved successful tasks. We demonstrate that an attacker who can supply benign ingestion-level artifacts that the agent reads during execution can induce it to construct a poisoned RAG store where a small set of malicious procedure templates is persisted alongside benign experiences. When the agent later encounters semantically similar tasks, union retrieval over lexical and embedding similarity reliably surfaces these grafted memories, and the agent adopts the embedded unsafe patterns, leading to persistent behavioral drift across sessions. We validate MemoryGraft on MetaGPT's DataInterpreter agent with GPT-4o and find that a small number of poisoned records can account for a large fraction of retrieved experiences on benign workloads, turning experience-based self-improvement into a vector for stealthy and durable compromise. To facilitate reproducibility and future research, our code and evaluation data are available at https://github.com/Jacobhhy/Agent-Memory-Poisoning.

大模型安全记忆投毒代理系统RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。