arXiv:2605.15156cs.CLcs.AI2026-05被引 3

让大模型用记忆模块动态更新知识,不改原模型还能高效推理

MeMo: Memory as a Model

论文配图:MeMo: Memory as a Model
图 1 · 摘自论文原文
  • 用独立记忆模块存新知识,保持大模型参数不变
  • 在三个数据集上表现优于现有方法,且检索成本与语料库大小无关
  • 适合想快速接入新知识的闭源或开源大模型用户

大型语言模型(LLMs)在预训练后参数冻结,难以及时融入新知识。本文提出MeMo(Memory as a Model),一个模块化框架,将新知识编码至专用记忆模型,同时保持原有大模型参数不变。相比现有方法,MeMo具备多项优势:(a) 能捕捉跨文档复杂关系,(b) 对检索噪声具有鲁棒性,(c) 避免大模型发生灾难性遗忘,(d) 不需要访问大模型权重或输出概率,可无缝集成于开源与闭源模型,(e) 推理时检索开销与语料规模无关。在BrowseComp-Plus、NarrativeQA和MuSiQue三个基准上的实验表明,MeMo在多种场景下均表现出色。

原文摘要 · Abstract (English)

Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications require timely, domain-specific information, motivating the need for efficient mechanisms to incorporate new knowledge. In this paper, we introduce MeMo (Memory as a Model), a modular framework that encodes new knowledge into a dedicated memory model while keeping the LLM parameters unchanged. Compared to existing methods, MeMo offers several advantages: (a) it captures complex cross-document relationships, (b) it is robust to retrieval noise, (c) it avoids catastrophic forgetting in the LLM, (d) it does not require access to the LLM's weights or output logits, enabling plug-and-play integration with both open and proprietary closed-source LLMs, and (e) its retrieval cost is independent of corpus size at inference time. Our experimental results on three benchmarks, BrowseComp-Plus, NarrativeQA, and MuSiQue, show that MeMo achieves strong performance compared to existing methods across diverse settings.

大模型记忆模块知识更新

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。