arXiv:2503.24007cs.LGcs.AI2025-03被引 7

提出CITRAS模型,高效融合时序预测中的可观测与已知协变量。

CITRAS: Covariate-Informed Transformer for Time Series Forecasting

  • 采用分块跨变量注意力机制,灵活整合目标变量与多种协变量。
  • 在多个真实数据集上优于主流模型,提升预测精度。
  • 适合需要融合外部因素的多变量时间序列预测任务。

在时间序列预测中,协变量代表影响目标变量的外部因素。部分协变量仅能过去观测(如气象数据),另一些则可提前获知(如日历事件或折扣安排)。尽管协变量有望提升预测性能,但多数深度学习模型难以处理因未来已知协变量导致的变量长度差异,且难以灵活利用。同时,准确捕捉目标变量与协变量间的依赖关系颇具挑战,需兼顾局部影响与全局跨变量依赖。为此,我们提出CITRAS——一种解码器仅的Transformer架构,可灵活整合多个目标变量、可观测协变量和已知协变量。在保持强自回归建模能力的同时,引入两种新机制:键值偏移(KV Shift)与注意力分数平滑。KV Shift通过基于并发依赖对齐未来已知协变量,无缝融入预测流程;注意力分数平滑通过平滑历史注意力分数,将局部精准的分块跨变量依赖升维为全局变量级依赖。实验表明,CITRAS在多种真实世界数据集上均表现优异,展现出在协变量引导与多变量场景下,有效利用跨变量与跨时间依赖以提升预测准确性的强大能力。

原文摘要 · Abstract (English)

In time series forecasting, covariates represent external factors that influence target variables. Some covariates are observable only in the past (observed covariates, such as recorded weather data), while others are known in advance (known covariates, such as calendar events or discount schedules). Although covariates have the potential to enhance forecasting performance, most deep learning-based forecasting models struggle to address the length discrepancy between variables caused by the future portion of known covariates and fail to leverage them flexibly. Moreover, capturing dependencies between target variables and covariates is non-trivial, as models must accurately reflect the local impact of covariates while simultaneously modeling global cross-variate dependencies. To address these challenges, we propose CITRAS, a decoder-only Transformer that flexibly integrates multiple target variables, observed covariates, and known covariates. While preserving strong autoregressive modeling capabilities, CITRAS introduces two novel mechanisms within patch-wise cross-variate attention: Key-Value (KV) Shift and Attention Score Smoothing. KV Shift seamlessly incorporates the future portion of known covariates into the forecasting process by aligning them with target variables based on their concurrent dependencies. Attention Score Smoothing refines locally accurate patch-wise cross-variate dependencies into global variate-level dependencies by smoothing the historical attention scores. Experimentally, CITRAS demonstrates strong performance across a wide range of real-world datasets in both covariate-informed and multivariate settings, showcasing its versatile ability to leverage cross-variate and cross-time dependencies for improved forecasting accuracy.

时间序列Transformer协变量多变量预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。