用检索替代长序列，让时间序列预测更准更快
Retrieval Mechanisms Surpass Long-Context Scaling in Time Series Forecasting
- 用可选检索替代无限上下文，只抓关键历史片段
- 3000步上下文使误差上升68%，证明长记不住有用信息
- 检索增强模型仅需720步就超越大模型，适合资源有限场景
时间序列基础模型（TSFMs）沿用自然语言处理的长上下文范式，假设输入更多历史数据能提升预测质量。但在随机领域中，遥远的历史往往只是高频噪声而非信号。本文通过在ETTh1基准上测试连续上下文架构（含PatchTST），发现结果与预设相反：出现明显逆向缩放规律，预测误差随上下文增长而上升，3000步窗口导致性能下降超过68%，表明注意力机制难以有效忽略无关波动。为此提出检索增强预测（RAFT），采用固定720步窗口与选择性检索，实现均方误差（MSE）0.379，优于长上下文配置及零样本基础模型（Chronos、Moirai），且计算量更低。检索步骤将最相关的历史片段作为动态外生变量注入，赋予模型无法从原始序列自建的上下文感知归纳偏置。因此，未来基础模型应转向以选择性检索为核心的架构设计。
原文摘要 · Abstract (English)
Time Series Foundation Models (TSFMs) have borrowed the long context paradigm from natural language processing under the premise that feeding more history into the model improves forecast quality. But in stochastic domains, distant history is often just high-frequency noise, not signal. Hence, the proposed work tests whether this premise actually holds by running continuous context architectures (PatchTST included) through the ETTh1 benchmark. The obtained results contradict the premise: an inverse scaling law shows up clearly, with forecasting error rising as context gets longer. A 3,000-step window causes performance to drop by over 68%, evidence that attention mechanisms are poor at ignoring irrelevant historical volatility. Retrieval-Augmented Forecasting (RAFT) is evaluated as an alternative. RAFT achieves a mean squared error (MSE) of 0.379 with a fixed 720-step window and selective retrieval, outperforming both long-context configurations and zero-shot foundation models (Chronos, Moirai) despite requiring far less computation. In addition, the retrieval step injects only the most relevant historical segments as dynamic exogenous variables, which gives the model a context-informed inductive bias it cannot build on its own from raw sequences. Therefore, foundation models going forward need to shift architecturally toward selective retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。