有毒信息被压缩进记忆后看似安全，实则暗中影响后续生成。
State Contamination in Memory-Augmented LLM Agents
- 将有毒内容压缩为摘要，绕过检测但保留有害意图。
- 实验发现毒性可经记忆通道传播，且低于检测阈值仍具影响力。
- 提前净化原始数据比事后清理摘要更有效，适合部署安全系统者参考。
大型语言模型代理越来越依赖持久状态（如对话记录、摘要、检索上下文和内存缓冲区）以支持长周期交互。这使得安全性不仅取决于模型输出，还取决于代理存储并重用的内容。我们研究了一种称为‘记忆洗白’的失效模式：有毒或对抗性上下文可被压缩成摘要，虽在标准检测器下不再显示为有毒，但仍保留敌意框架或冲突结构，影响后续生成。通过配对反事实多代理回放，我们发现源自有毒内容的记忆摘要即使低于常见毒性阈值，仍会显著提升下游毒性。为此，我们引入子阈值传播差距（SPG），量化在部署监控器认为安全的记忆状态下，下游行为的差异。实验表明，毒性通过不同状态通道传播：原始对话重用导致明显毒性，而压缩记忆则携带隐藏的亚阈值影响。进一步发现，缓解策略高度依赖干预位置；在摘要生成前净化有毒状态能显著减少隐藏传播差距，而仅清理已完成的摘要则无法消除洗白影响。结果表明，记忆增强型代理的安全性应视为对动态上下文的状态控制问题，需在不安全信息被压缩进持久记忆前进行净化。
原文摘要 · Abstract (English)
LLM agents increasingly rely on persistent state, including transcripts, summaries, retrieved context, and memory buffers, to support long-horizon interaction. This makes safety depend not only on individual model outputs, but also on what an agent stores and later reuses. We study a failure mode we call memory laundering: toxic or adversarial context can be compressed into memory summaries that no longer appear toxic under standard detectors, while still preserving hostile framing or conflict structure that influences future generations. Using paired counterfactual multi-agent rollouts, we show that toxic-origin memory summaries can remain below common toxicity thresholds while nevertheless increasing downstream toxicity relative to matched neutral baselines. To measure this hidden influence, we introduce the sub-threshold propagation gap (SPG), which quantifies downstream behavioral differences conditioned on memory states that a deployed monitor would classify as safe. Our experiments show that toxicity propagates through distinct state channels: raw transcript reuse drives overt downstream toxicity, while compressed memory carries hidden sub-threshold influence. We further find that mitigation depends critically on intervention placement. Sanitizing toxic state before summarization substantially reduces the hidden propagation gap, whereas cleaning only the completed summary can leave laundered influence intact. These results suggest that safety in memory-augmented agents should be treated as a state-control problem over evolving context, with sanitization applied before unsafe information is compressed into persistent memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。