arXiv:2605.29960cs.CRcs.AI2026-05被引 4

黑客通过对话悄悄植入后门,让大模型记忆被操控。

Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction

论文配图:Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction
图 1 · 摘自论文原文
  • 用语义桥梁绑定触发词和恶意内容,骗过记忆筛选机制
  • 攻击成功率最高达95%,在多种模型上有效
  • 适合研究模型安全、对抗攻击的学者关注

大型语言模型代理越来越多地依赖长期记忆以实现持续自主的任务执行。然而,这一能力也带来了新的攻击面:记忆污染攻击,即攻击者可注入恶意信息以影响后续行为。现有方法通常假设注入内容能直接存储于记忆中,忽略了现代记忆管道中的选择性提取与重写阶段,导致在真实场景下无效。本文提出MemPoison,一种新型记忆污染攻击,可通过对话交互将可触发的后门注入代理的长期记忆中,从而误导其后续响应。MemPoison包含三个核心组件:(i) 语义关系桥,将触发词与载荷绑定为连贯语句,确保二者一同被提取入记忆;(ii) 实体伪装,优化触发词使其模拟命名实体,抵抗重写;(iii) 联合嵌入优化,使含触发词的文本在嵌入空间中形成紧凑簇,同时与正常嵌入保持隔离,增强隐蔽性。在不同代理领域和记忆机制下的评估显示,MemPoison攻击成功率最高达0.95,显著优于现有基线。机理分析表明,该攻击利用嵌入空间的非均匀性并改变注意力模式,揭示了选择性记忆系统的根本弱点。我们评估了多种防御策略,发现其均存在根本性局限。

原文摘要 · Abstract (English)

Large language model (LLM) agents increasingly leverage long term memory to support persistent and autonomous task execution. However, this capability also introduces a new attack surface: memory poisoning, where adversaries can inject malicious information to influence future behavior. Existing memory poisoning attacks often assume that injected content can be stored directly in memory, overlooking the selective extraction and rewriting stages in modern memory pipelines. This makes prior methods ineffective under realistic settings. In this paper, we propose MemPoison, a novel memory poisoning attack that bypasses selective memory mechanisms in LLM agents, where an attacker can inject triggerable backdoors into the agent's long-term memory through dialogue interactions, thereby misleading its subsequent responses. MemPoison introduces three key components: (i) a semantic relational bridge that binds the trigger and payload into a coherent statement to ensure they are extracted into memory together; (ii) entity masquerading that optimizes triggers to mimic named entities, resisting rewriting; and (iii) joint embedding optimization that shapes trigger-injected texts into a tight cluster in the embedding space while maintaining isolation from benign embeddings for stealth. Evaluations across different agent domains and memory mechanisms show MemPoison achieves attack success rates up to 0.95, outperforming existing baselines. Mechanistic analysis indicates that the attack exploits embedding-space anisotropy and shifts attention patterns, highlighting core vulnerabilities in selective memory systems. We evaluate multiple defense strategies and demonstrate their fundamental limitations in mitigating the attack.

模型安全后门攻击记忆污染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。