arXiv:2607.26760cs.CLcs.LG2026-07被引 2

让大模型自带记忆能力,无需外部模块即可自动存取历史信息。

Metis: Memory Foundation Model

论文配图:Metis: Memory Foundation Model
图 1 · 摘自论文原文
  • 在模型内部构建可动态演化的真实记忆状态,通过计算自主存储与调用。
  • 训练时仅需前向传播即可更新记忆,推理时模型权重冻结但记忆自更新。
  • 首次实现无需外部记忆模块的原生记忆大模型,适合智能体与长期交互任务。

近年来,人工智能代理逐渐将原生能力内化至基础模型中,催生了多模态基础模型和大推理模型。然而,代理记忆仍主要依赖外部模块,原生记忆能力尚未被充分探索。本文首次提出记忆基础模型,赋予基础模型原生记忆能力。从两个角度形式化原生记忆:一是骨干网络中的持久化、动态演化的记忆状态;二是通过模型计算自主完成信息存储与利用的原生记忆机制。我们证明原生记忆在架构、端到端优化和效率方面具有优势。基于此,提出首个原型模型Metis,引入新架构使基础模型具备原生记忆状态,历史信息被压缩并经由记忆注意力访问。我们构建大规模记忆专用训练数据,引入多种优化目标,在训练中期学习这些原生记忆机制。Metis的在线记忆维护无需梯度,更新仅需前向传播。推理时所有模型权重保持冻结,而原生记忆状态通过标准前向计算自动演化。大量实验表明,Metis展现出原生记忆能力,并对其优势、局限和行为进行了详细分析。为推动未来研究,项目及模型检查点已公开。

原文摘要 · Abstract (English)

Recent advances in AI agents have increasingly internalized native capabilities into their underlying foundation models, giving rise to multimodal foundation models and large reasoning models. However, agent memory is still primarily implemented through external modules, leaving the native memory capability largely unexplored. In this paper, we take a first step toward this direction by introducing memory foundation models, which empower foundation models with native memory capabilities. We formalize native memory from two perspectives: a persistent and dynamically evolving memory state within the backbone, and native memory procedures that autonomously store and utilize information through model computation. We show that native memory offers advantages in architecture, end-to-end optimization, and efficiency. Based on this formulation, we propose Metis, the first prototype of memory foundation models. Metis introduces a new architecture that equips a foundation model with a native memory state, allowing historical information to be compressed into the model and accessed through memory attention. We construct large-scale memory-specific training data and introduce multiple optimization objectives to acquire these native memory procedures through mid-training. The online memory maintenance of Metis is gradient-free, and the memory update requires only a forward pass. At inference time, all learned model weights remain frozen, while the native memory states are autonomously transformed through standard forward computation. Through extensive experiments, we show that Metis exhibits native memory capabilities and further provide a detailed analysis of its strengths, limitations, and behaviors. To facilitate future research on memory foundation models, we release our project and model checkpoints.

记忆模型基础模型智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。