arXiv:2509.18868cs.AI2025-09被引 11

给大模型记忆建立统一评估框架,可测可管可审计。

Memory in Large Language Models: Mechanisms, Evaluation and Evolution

  • 按写入位置和访问方式分四类记忆,构建可量化评估体系。
  • 提出三阶段测试协议,避免不同场景下结果失真。
  • 支持记忆更新、遗忘与安全审计,适合研究与部署使用。

在统一操作定义下,我们将大语言模型的记忆定义为在预训练、微调或推理过程中写入的持久状态,可后续被访问并稳定影响输出。提出四分类记忆体系(参数化、上下文、外部、程序/情景)和记忆四元组(位置、持续性、读写路径、可控性)。通过写->读->抑制/更新链条,贯通机制、评估与治理。为避免异构设置导致的比较偏差,采用三设定协议(仅参数化、离线检索、在线检索),分离能力与信息可用性。在此基础上构建分层评估:参数化(闭卷召回、编辑差异、记忆/隐私)、上下文(位置曲线、中段下降)、外部(答案正确性与片段归属/忠实度)、程序/情景(跨会话一致性与时间线重播,E MARS+)。整合时间治理与泄漏审计(新鲜度命中、过时回答、拒绝切片)及不确定性报告(评分者一致性+多重比较校正配对检验)。针对更新与遗忘,提出DMM Gov:协调DAPT/TAPT、PEFT、模型编辑(ROME、MEND、MEMIT、SERAC)与RAG,形成可审计闭环,覆盖准入阈值、发布、监控、回滚与变更审计,明确时效性、冲突处理与长周期一致性要求。最后给出四项可验证命题:最小可识别性;最小评估卡;因果约束编辑与可验证遗忘;小窗口重播优于超长上下文阅读的场景。构建了一个可复现、可比较、可治理的研究与部署坐标系。

原文摘要 · Abstract (English)

Under a unified operational definition, we define LLM memory as a persistent state written during pretraining, finetuning, or inference that can later be addressed and that stably influences outputs. We propose a four-part taxonomy (parametric, contextual, external, procedural/episodic) and a memory quadruple (location, persistence, write/access path, controllability). We link mechanism, evaluation, and governance via the chain write -> read -> inhibit/update. To avoid distorted comparisons across heterogeneous setups, we adopt a three-setting protocol (parametric only, offline retrieval, online retrieval) that decouples capability from information availability on the same data and timeline. On this basis we build a layered evaluation: parametric (closed-book recall, edit differential, memorization/privacy), contextual (position curves and the mid-sequence drop), external (answer correctness vs snippet attribution/faithfulness), and procedural/episodic (cross-session consistency and timeline replay, E MARS+). The framework integrates temporal governance and leakage auditing (freshness hits, outdated answers, refusal slices) and uncertainty reporting via inter-rater agreement plus paired tests with multiple-comparison correction. For updating and forgetting, we present DMM Gov: coordinating DAPT/TAPT, PEFT, model editing (ROME, MEND, MEMIT, SERAC), and RAG to form an auditable loop covering admission thresholds, rollout, monitoring, rollback, and change audits, with specs for timeliness, conflict handling, and long-horizon consistency. Finally, we give four testable propositions: minimum identifiability; a minimal evaluation card; causally constrained editing with verifiable forgetting; and when retrieval with small-window replay outperforms ultra-long-context reading. This yields a reproducible, comparable, and governable coordinate system for research and deployment.

大模型记忆评估框架可审计性模型治理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。