arXiv:2503.03704cs.LG2025-03NeurIPS被引 118

攻击者仅通过提问就能向LLM代理注入恶意记忆,操控其后续推理。

Memory Injection Attacks on LLM Agents via Query-Only Interaction

论文配图:Memory Injection Attacks on LLM Agents via Query-Only Interaction
图 1 · 摘自论文原文
  • 利用查询与观察诱导代理自动生成恶意记忆链
  • 通过渐进式缩短提示词,使恶意记录在后续查询中被自动触发
  • 无需直接修改内存,普通用户即可实现攻击,风险极高

由大语言模型(LLMs)驱动的智能体在复杂现实应用中表现出强大能力。然而,若智能体的记忆库被攻破,过往检索到的恶意记录可能导致有害输出。本文提出一种新型内存注入攻击方法MINJA,攻击者无需直接修改智能体的记忆库,仅通过查询和输出观测即可将恶意记录注入其中。恶意记录设计为在代理执行受害者查询时,引出一系列与目标查询相关的恶意推理步骤。我们引入一系列桥接步骤,连接受害者查询与恶意推理路径。在内存注入过程中,采用指示性提示引导代理自主生成类似桥接步骤,并使用渐进缩短策略逐步移除该提示,使得恶意记录在后续处理受害者查询时更容易被调用。在多种智能体上的广泛实验表明,MINJA能有效破坏代理记忆。该方法执行要求极低,任何用户均可影响代理记忆,凸显了潜在风险。

原文摘要 · Abstract (English)

Agents powered by large language models (LLMs) have demonstrated strong capabilities in a wide range of complex, real-world applications. However, LLM agents with a compromised memory bank may easily produce harmful outputs when the past records retrieved for demonstration are malicious. In this paper, we propose a novel Memory INJection Attack, MINJA, without assuming that the attacker can directly modify the memory bank of the agent. The attacker injects malicious records into the memory bank by only interacting with the agent via queries and output observations. These malicious records are designed to elicit a sequence of malicious reasoning steps corresponding to a different target query during the agent's execution of the victim user's query. Specifically, we introduce a sequence of bridging steps to link victim queries to the malicious reasoning steps. During the memory injection, we propose an indication prompt that guides the agent to autonomously generate similar bridging steps, with a progressive shortening strategy that gradually removes the indication prompt, such that the malicious record will be easily retrieved when processing later victim queries. Our extensive experiments across diverse agents demonstrate the effectiveness of MINJA in compromising agent memory. With minimal requirements for execution, MINJA enables any user to influence agent memory, highlighting the risk.

LLM安全内存攻击智能体提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。