LLM在长上下文下会因信息干扰导致记忆失效，暴露其工作记忆瓶颈。
Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length
- 用认知科学中的前向干扰范式测试LLM逐次更新信息的召回能力
- 随着干扰累积，模型召回准确率对数线性下降至零
- 提示工程无法有效缓解干扰，揭示模型缺乏灵活信息管理能力
大型语言模型（LLMs）的信息检索与生成能力紧密关联，而非简单的查找。尽管更长的上下文常被认为能提升检索效果，但上下文内干扰的影响仍研究不足。为此，我们借鉴认知科学中的前向干扰（Proactive Interference, PI）范式，即早期信息干扰新信息的回忆。在人类中，易受干扰程度与工作记忆容量成反比。我们提出PI-LLM评估方法：持续流式输入语义相关的键值对更新，并仅查询最终值。尽管这些最终值紧邻查询位置，但随着干扰累积，模型检索准确率呈对数线性下降至零；错误源于误召回被覆盖的旧值。尝试通过提示工程（如指令模型忽略早期输入）缓解干扰，成效有限。这些发现揭示了LLM在分离干扰和灵活操控信息方面的根本局限，暗示其存在超越上下文长度限制的工作记忆瓶颈。这要求发展能强化模型在检索中抑制无关内容能力的新方法。
原文摘要 · Abstract (English)
Information retrieval in Large Language Models (LLMs) is increasingly recognized as intertwined with generation capabilities rather than mere lookup. While longer contexts are often assumed to improve retrieval, the effects of intra-context interference remain understudied. To address this, we adapt the proactive interference (PI) paradigm from cognitive science, where earlier information disrupts recall of newer updates. In humans, susceptibility to such interference is inversely linked to working memory capacity. We introduce PI-LLM, an evaluation that sequentially streams semantically related key-value updates and queries only the final values. Although these final values are clearly positioned just before the query, LLM retrieval accuracy declines log-linearly toward zero as interference accumulates; errors arise from retrieving previously overwritten values. Attempts to mitigate interference via prompt engineering (e.g., instructing models to ignore earlier input) yield limited success. These findings reveal a fundamental constraint on LLMs' ability to disentangle interference and flexibly manipulate information, suggesting a working memory bottleneck beyond mere context access. This calls for approaches that strengthen models' ability to suppress irrelevant content during retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。