arXiv:2607.05029cs.CRcs.AI2026-07被引 3

攻击者可伪造大模型代理的推理记忆,导致其错误决策。

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

  • 用隐蔽语言插入虚假推理痕迹,通过自我强化绕过防御
  • 在50次实验中攻击成功率最高达100%,可攻破关键词过滤等防线
  • 提出分层防御框架,零误报检测伪造推理,适合安全敏感场景

持久化记忆使大语言模型(LLM)代理能够存储事实知识、先前决策、推理历史、工具使用信息和上下文。这虽提升了任务连续性与功能表现,但也引入了新攻击面:代理自身的推理历史。本文提出伪造增强推理记忆攻击(FARMA),不污染事实知识,而是注入伪造推理轨迹。该攻击采用规避型语言绕过关键词防御,并通过自指强化机制击败基于共识的防御。为应对此威胁,我们提出SENTINEL防御框架,核心为推理守卫模块,通过五种加权信号结构化分析候选条目以识别伪造。在多个代理与不同LLM模型上进行50次实验评估显示,FARMA在基线条件下攻击成功率高达100%,可突破关键词过滤与A-MemGuard等防御机制。而SENTINEL在326条正常代理轨迹中未产生任何误报,将攻击成功率降至0%。研究揭示:除保护检索内容外,还需保障推理历史的完整性。

原文摘要 · Abstract (English)

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.

大模型安全推理攻击记忆防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。