研究大模型智能体记忆如何被攻击，发现只需篡改记忆就能误导答题。
Memory as an Attack Surface in LLM Agents: A Study on Multiple-Choice Question Answering
- 构建带外部记忆的智能体，通过注入错误信息测试其抗干扰能力。
- 篡改记忆后准确率下降显著，错误选项被选中的比例明显上升。
- 揭示记忆是新型攻击面，对需高可靠性的系统有警示意义。
AI智能体通过融合语言理解、任务执行与记忆机制，扩展了传统大语言模型的应用。虽然记忆有助于提供个性化和上下文相关的响应，但也带来了新威胁：存储在记忆中的信息可能影响后续输出，即使当前查询干净无误。本文研究基于大语言模型的智能体在多选题问答中受到的记忆操纵问题。我们设计并实现了一个带有外部记忆组件的智能体，用于存储和检索任务相关的信息。随后，在智能体回答多选题前，引入基础的记忆操纵场景，向其内存中注入误导或损坏的信息。在受控实验环境下，对比记忆操纵前后智能体的表现，测量答案准确率、攻击成功率及被操纵选项的选择比例。结果表明，即使是简单的记忆操纵，也能显著影响智能体最终的答案，导致其在接收到清晰完整的问题时仍选择错误选项。
原文摘要 · Abstract (English)
AI agents extend conventional large language model (LLM) applications by integrating language understanding with task execution, external tool use, and memory mechanisms. While memory allows agents to retain prior interactions and provide more personalized and context-aware responses, it also introduces a new vulnerability: information stored in memory can influence future outputs even when the current query is clean. In this paper, we investigate memory manipulation in LLM-based agents for multiple-choice question answering. We first design and implement an LLM-based AI agent with an external memory component that stores and retrieves task-relevant information. We then introduce basic memory manipulation scenarios in which misleading or corrupted memories are inserted into the agent before it answers multiple-choice questions. Using a controlled experimental setup, we compare the agent's performance before and after memory manipulation and measure changes in answer accuracy, attack success rate, and selection of manipulated options. Our results show that even simple memory manipulations can noticeably affect the agent's final answers, causing it to select incorrect options despite receiving clean and well-formed questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。