让时间信息深度参与模型推理，显著提升长期时序预测精度
Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting
- 在多个层级用可学习的时间令牌，通过交叉注意力注入时间先验
- 仅训练交叉注意力模块，在多个数据集上达到最新最优的长期预测效果
- 方法轻量高效，适合需要精准时间推理的工业级时序建模场景
基于大语言模型的时序预测方法通常对时间处理浅层化，仅在输入端注入位置或提示信息，且解码器基本固定，导致时间信息在深层逐渐退化。本文提出时间先验条件化（TPC），将时间视为首要模态，在多个深度层级进行条件化。TPC在补丁流中附加一组可学习的时间序列标记，选定层中这些标记与由同一冻结大模型编码的紧凑、人类可读的时间描述符生成的时序嵌入进行交叉注意力，并通过自注意力将时间上下文反馈。该方法分离了时序信号与时间信息，同时保持参数开销低。实验表明，仅训练交叉注意力模块并显式分离时序信号与时间信息，即可持续优于全量微调和浅层条件化策略，在多种数据集上的长期预测任务中达到当前最优性能。代码已公开于：https://github.com/fil-mp/Deep_tpc
原文摘要 · Abstract (English)
LLM-for-time series (TS) methods typically treat time shallowly, injecting positional or prompt-based cues once at the input of a largely frozen decoder, which limits temporal reasoning as this information degrades through the layers. We introduce Temporal-Prior Conditioning (TPC), which elevates time to a first-class modality that conditions the model at multiple depths. TPC attaches a small set of learnable time series tokens to the patch stream; at selected layers these tokens cross-attend to temporal embeddings derived from compact, human-readable temporal descriptors encoded by the same frozen LLM, then feed temporal context back via self-attention. This disentangles time series signal and temporal information while maintaining a low parameter budget. We show that by training only the cross-attention modules and explicitly disentangling time series signal and temporal information, TPC consistently outperforms both full fine-tuning and shallow conditioning strategies, achieving state-of-the-art performance in long-term forecasting across diverse datasets. Code available at: https://github.com/fil-mp/Deep_tpc
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。