arXiv:2507.03724cs.CL2025-07被引 111

为大模型设计内存操作系统,实现长期记忆管理

MemOS: A Memory OS for AI System

  • 构建统一内存系统,整合文本、激活态和参数级记忆
  • 通过MemCube单元实现记忆的版本控制与跨类型迁移
  • 适合需要持续学习和个性化的大模型研究者

大语言模型已成为通用人工智能的关键基础设施,但其缺乏明确的记忆管理系统,制约了长上下文推理、持续个性化与知识一致性的发展。现有模型主要依赖静态参数和短暂上下文状态,难以追踪用户偏好或长期更新知识。虽然检索增强生成(RAG)引入了外部文本知识,但仍为无状态方案,缺乏生命周期管理和与持久表示的融合。近期研究从内存层次结构视角建模了大模型的训练与推理成本,表明在参数内存与外部检索之间引入显式记忆层,可显著降低这些开销。除计算效率外,信息随时间和上下文分布带来的挑战,也要求系统具备管理异构知识的能力,覆盖不同时间尺度与来源。为此,我们提出MemOS——一种将记忆视为可管理资源的内存操作系统。它统一了纯文本、基于激活态及参数级记忆的表示、调度与演化,支持高效存储与检索。作为基本单元,MemCube封装了记忆内容及其元数据(如来源、版本)。MemCube可随时间组合、迁移与融合,实现记忆类型的灵活转换,并连接检索与参数化学习。MemOS建立了以记忆为中心的系统框架,赋予大模型可控性、可塑性与可进化性,为持续学习与个性化建模奠定基础。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the development of long-context reasoning, continual personalization, and knowledge consistency.Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.While Retrieval-Augmented Generation (RAG) introduces external knowledge in plain text, it remains a stateless workaround without lifecycle control or integration with persistent representations.Recent work has modeled the training and inference cost of LLMs from a memory hierarchy perspective, showing that introducing an explicit memory layer between parameter memory and external retrieval can substantially reduce these costs by externalizing specific knowledge. Beyond computational efficiency, LLMs face broader challenges arising from how information is distributed over time and context, requiring systems capable of managing heterogeneous knowledge spanning different temporal scales and sources. To address this challenge, we propose MemOS, a memory operating system that treats memory as a manageable system resource. It unifies the representation, scheduling, and evolution of plaintext, activation-based, and parameter-level memories, enabling cost-efficient storage and retrieval. As the basic unit, a MemCube encapsulates both memory content and metadata such as provenance and versioning. MemCubes can be composed, migrated, and fused over time, enabling flexible transitions between memory types and bridging retrieval with parameter-based learning. MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to LLMs, laying the foundation for continual learning and personalized modeling.

大模型记忆系统持续学习内存管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。