长上下文窗口能降低预测不确定性,提升时间序列建模精度。
Why Do Time Series Models Need Long Context Windows?

- 将预测拆解为过程识别与条件预测两任务,揭示长窗口作用机制。
- 证明即使记忆长度为P,窗口必须大于P才能达到最优误差。
- 解耦两任务可提升计算效率,适合大规模时间序列系统设计。
现代多时间序列预测模型依赖越来越长的观测窗口,但其优势常被归因于捕捉长程依赖,对全局模型如何利用输入信息的讨论较少。本文指出,时间序列预测包含两个目标:(i) 生成过程识别(GPI),即推断输入序列的生成机制;(ii) 条件预测(CF),即基于输入观测预测未来值。最优预测可解释为在给定输入窗口下,对可能生成过程的加权平均,权重为其似然性。这提供了一个新解释:长上下文窗口能降低对生成过程的不确定性。我们证明,即使真实过程记忆长度为P,输入窗口严格大于P时才能达到最小可实现误差。最后,我们展示解耦GPI与CF可在不损失精度的前提下提升计算可扩展性。合成与真实数据实验验证了这些洞见对模型架构设计的重要性。
原文摘要 · Abstract (English)
Modern deep learning models for forecasting groups of time series rely on increasingly longer observation windows. However, the benefit of increasing the window size is often simply attributed to capturing long-range dependencies, and broader discussion on how global forecasting models leverage input observations has been limited. In this paper, we show that forecasting groups of time series involves two objectives: (i) generative process identification (GPI), i.e., inferring the specific process generating the input sequence, and (ii) conditional forecasting (CF), i.e., predicting future values given input observations. From this perspective, optimal predictions can be interpreted as an average over plausible data-generating processes, weighted by their likelihood given the input window. This suggests another explanation for the benefits of long context windows: they reduce the uncertainty about which specific process is generating the input time series during operation. We prove that even for processes with memory length $P$, an input window size strictly larger than $P$ is necessary to achieve the minimum attainable error. Finally, we show how decoupling GPI and CF can improve computational scalability without compromising accuracy. Experiments on synthetic and real-world data validate our insights and their relevance for designing forecasting architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。