提出统一框架,系统分析时序模型如何记忆与调用长期信息。
Memory in Deep Time-Series Models

- 从记忆角度重新梳理时序模型演进,分内部与外部记忆机制。
- 识别出三类外部记忆:显式模块、检索增强、智能体存储。
- 为长期依赖建模提供新视角,适合研究时序长程推理的学者。
时序建模的深度学习发展经历了从循环网络、Transformer到结构化状态空间模型、检索增强预测器、基础模型及工具使用智能体等范式更迭。这些进展常被孤立研究,按架构或时代划分。本文主张以核心问题重审:时序模型如何保留并访问超出即时输入的信息?传统方法受限于输入窗口,压缩历史为固定大小状态易丢失未来有用信息。为此,将挑战定义为‘记忆’问题,构建从参数编码的内部记忆到代理维护的可寻址、可检索外部记忆的连续谱系。提出统一分类框架,涵盖三类外部记忆机制:显式模块、检索增强、智能体存储,并统一分析其保留内容、写入与访问方式及持久性。跨任务分析揭示方法与评估的空白。最后指出构建可选择性保留、检索、修订与遗忘信息的记忆系统的开放问题。该框架使记忆成为独立于主干模型的第一性维度。
原文摘要 · Abstract (English)
Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state-space models, retrieval-augmented predictors, foundation models, and tool-using agents. These developments are typically studied in isolation, organized by architecture or modeling era. We argue that they can instead be viewed through a common question of \emph{how does a time-series model retain and access information beyond its immediate input?} This question is motivated by a fundamental limitation of conventional time-series modeling: information relevant to a prediction may lie far beyond a feasible input window, while compressing history into a fixed-size state can discard information that may become useful later. We formulate this challenge as a \emph{memory} problem and organize existing time-series methods along a spectrum from internal memory, encoded in parameters and fixed-size states, to external memory that is addressable, retrievable, and increasingly maintained by agents. We then develop a unified taxonomy of memory mechanisms and review three classes of external memory, including explicit modules, retrieval augmentation, and agentic stores, under a common framework for what is retained, how it is written and accessed, and how it persists. A cross-cutting analysis maps these mechanisms to time-series tasks and identifies gaps in both methods and evaluation. We conclude by outlining open problems in building memory systems that can selectively retain, retrieve, revise, and forget information as temporal environments evolve. The result is a framework for studying memory as a first-class dimension of time series modeling, independent of the underlying backbone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。