arXiv:2605.23723cs.AI2026-05被引 3

发现并定位恶意记忆,让中毒的AI-agent能自查自纠。

MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection

论文配图:MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection
图 1 · 摘自论文原文
  • 用因果分析评估每条记忆对错误输出的影响程度。
  • 通过结构异常检测找出被篡改的记忆节点,攻击成功率降至0%。
  • 适合关注AI安全、内存污染防御的研究者与开发者。

大型语言模型代理越来越依赖持久化记忆来存储过往交互、检索相关示例,并提升长程任务执行能力。然而,这一机制也带来实际安全风险:攻击者可通过正常交互向代理内存中注入恶意记录,这些记录后续可能被调用以操控代理推理与行为。现有防御多聚焦于在线干预(如提示过滤或输出拦截),但无法解决有害行为发生后的事后溯源问题。本文提出 extbf{MemAudit},一种面向记忆增强型LLM代理的事后因果记忆审计框架。该框架结合两种互补信号:(1) 反事实记忆影响得分,用于衡量每条记忆对有害输出的因果贡献;(2) 记忆一致性图,识别内存库中结构异常的记忆项。我们在 MINJA 攻击场景下进行评估,该攻击仅通过查询方式注入恶意记忆,不直接修改内存。在问答与推理代理设置中,MemAudit 在真实事后审计条件下显著降低攻击成功率:问答攻击成功率从70%降至0%,RAP攻击从83.3%降至0%。

原文摘要 · Abstract (English)

Large language model agents increasingly rely on persistent memory to store past interactions, retrieve relevant demonstrations, and improve long-horizon task execution. However, this memory mechanism also creates a practical security vulnerability: an adversarial user may inject malicious records into the agent's memory through ordinary interaction, and these records can later be retrieved to steer the agent's reasoning and actions. Existing defenses primarily focus on online intervention, such as prompt filtering or output blocking, but they do not address the post-hoc question of which stored memories are responsible after harmful behavior has already been observed. We propose \textbf{MemAudit}, a post-hoc causal memory auditing framework for memory-augmented LLM agents. The framework combines two complementary signals: (1) a counterfactual memory influence score that measures each memory's causal contribution to harmful outputs, and (2) a memory consistency graph that identifies structurally anomalous memories within the broader memory store. We evaluate MemAudit against MINJA, a query-only memory injection attack in which malicious records are generated and stored through normal agent interactions rather than direct memory-bank modification. Across both QA and reasoning-agent settings, MemAudit substantially reduces attack success rates under realistic post-hoc auditing scenarios. The results show that QA attack success is reduced from $70\%$ to $0\%$, while RAP attack success drops from $83.3\%$ to $0\%$.

AI安全记忆审计因果分析对抗攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。