让语言模型像版本控制一样高效更新知识,无需重训。
Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo
- 用版本感知操作层实现知识的精准编辑与回滚。
- 通过事务记忆机制支持多步修改,可追溯、可复用。
- 适合需要动态更新知识的长期运行模型系统。
MeMo 提出具有显式多层相关矩阵记忆(CMMs)的语言模型,其中记忆、检索和遗忘是架构级操作。本文探讨当知识变化时,如何减少对重新训练的需求。对于可表达为 MeMo 记忆关联的变化,可通过编辑显式记忆而非重训整个模型来更新模型的知识。我们提出一个版本感知操作层,将替换、失效、保留历史、回滚和追踪等高层操作编译为针对序列和标记的 MeMo 原语调用。关键观察是:版本感知操作通常不是单一记忆关联,而是一系列有序的原始编辑事务,例如遗忘一条序列-标记链,记忆另一条,保留历史链,并记录反向程序。该框架引入两个辅助 CMM:版本 CMM(V-CMM)用于将版本转换映射到事务句柄,事务 CMM(T-CMM)用于存储可复用的变更内容和反向程序。支持直接序列级编辑和结构化 diff 级输入,并规划了评估更新成功率、回滚能力、可追溯性、局部性和事务复用性的路径。
原文摘要 · Abstract (English)
MeMo proposes language models with explicit multi-layer correlation matrix memories (CMMs), where memorization, retrieval, and forgetting are architectural operations. This paper asks how such memories can reduce the need for retraining when knowledge changes. For changes expressible as MeMo memory associations, the model's accessible knowledge can be updated by editing explicit memories rather than retraining the whole model. We propose a version-aware operation layer in which high-level operations such as replace, obsolete, keep-history, rollback, and trace are compiled into MeMo-native primitive calls over sequences and tokens. The key observation is that a version-aware operation is rarely a single MeMo association. It is an ordered transaction of primitive edits, for example forgetting one sequence-token chain, memorizing another, preserving a historical chain, and recording an inverse program. The framework introduces two auxiliary CMMs: a Version CMM (V-CMM) for mapping version transitions to transaction handles, and a Transaction CMM (T-CMM) for storing reusable change contents and inverse programs. It supports both direct sequence-level edits and structured diff-level inputs, and outlines an evaluation route for update success, rollback, traceability, locality, and transaction reuse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。