arXiv:2601.05505cs.CL2026-01ACL被引 6

让大模型自动记住对话历史,速度提升5倍

FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse

论文配图:FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse
图 1 · 摘自论文原文
  • 从推理过程的中间状态提取记忆,不需额外模型
  • 在相同性能下推理延迟降低5倍
  • 适合需要长时间对话的智能体应用

大型语言模型的无状态架构缺乏动态上下文保留机制,导致智能体必须重复处理历史信息以维持长时自主性。现有隐式记忆方法受限于架构分离,依赖独立编码器将记忆与推理主干解耦。本文提出FlashMem框架,通过计算复用从临时推理状态中蒸馏内在记忆。利用内部表征唯一编码输入轨迹的特性,FlashMem将最后一个隐藏状态视为交互历史的充分统计量,使共享键值聚合器直接访问冻结缓存生成记忆,避免冗余重参数化。此外,一个无参数认知监控器通过注意力熵自适应触发记忆整合,仅在高认知不确定性时激活。实验表明,FlashMem在保持重型基线性能的同时,推理延迟降低5倍,有效弥合效率与持续认知之间的差距。

原文摘要 · Abstract (English)

The stateless architecture of Large Language Models inherently lacks the mechanism to preserve dynamic context, compelling agents to redundantly reprocess history to maintain long-horizon autonomy. While latent memory offers a solution, current approaches are hindered by architectural segregation, relying on auxiliary encoders that decouple memory from the reasoning backbone. We propose FlashMem, a framework that distills intrinsic memory directly from transient reasoning states via computation reuse. Leveraging the property that internal representations uniquely encode input trajectories, FlashMem identifies the last hidden state as a sufficient statistic for the interaction history. This enables a Shared-KV Consolidator to synthesize memory by attending directly to the backbone's frozen cache, eliminating redundant re-parameterization. Furthermore, a parameter-free Cognitive Monitor leverages attention entropy to adaptively trigger consolidation only when high epistemic uncertainty is detected. Experiments demonstrate that FlashMem matches the performance of heavy baselines while reducing inference latency by 5 times, effectively bridging the gap between efficiency and persistent cognition.

大模型记忆推理优化智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。