arXiv:2410.12184cs.LG2024-10中稿 · ICDM 2024被引 8

用Transformer融合历史与当前外部变量,提升长期时间序列预测精度

ExoTST: Exogenous-Aware Temporal Sequence Transformer for Time Series Prediction

  • 设计跨时模态融合模块,分别处理历史与当前外部变量
  • 在碳通量等真实数据上最高提升10%预测准确率
  • 对缺失值和噪声有强鲁棒性,适合实际应用

精准的长期预测是众多机器学习应用和决策过程的基础。传统时间序列方法通常只依赖目标变量的历史自回归(内生变量)或仅考虑当前协变量驱动(外生变量),但如何有效结合历史内生变量、历史外生变量与当前外生变量仍是挑战。本文提出ExoTST,一种基于Transformer的新框架,通过注意力机制引入新颖的跨时模态融合模块,将历史与当前外生序列作为不同模态进行联合建模。该设计提升了模型对历史与当前外生变量分布偏移的鲁棒性。在真实碳通量数据集和时间序列基准测试上的实验表明,ExoTST相比现有最优模型性能提升高达10%。此外,其在存在缺失值和噪声的外生驱动下仍保持稳定表现,具备强实用性。

原文摘要 · Abstract (English)

Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. Traditional time series approaches for prediction often focus on either autoregressive modeling, which relies solely on past observations of the target ``endogenous variables'', or forward modeling, which considers only current covariate drivers ``exogenous variables''. However, effectively integrating past endogenous and past exogenous with current exogenous variables remains a significant challenge. In this paper, we propose ExoTST, a novel transformer-based framework that effectively incorporates current exogenous variables alongside past context for improved time series prediction. To integrate exogenous information efficiently, ExoTST leverages the strengths of attention mechanisms and introduces a novel cross-temporal modality fusion module. This module enables the model to jointly learn from both past and current exogenous series, treating them as distinct modalities. By considering these series separately, ExoTST provides robustness and flexibility in handling data uncertainties that arise from the inherent distribution shift between historical and current exogenous variables. Extensive experiments on real-world carbon flux datasets and time series benchmarks demonstrate ExoTST's superior performance compared to state-of-the-art baselines, with improvements of up to 10\% in prediction accuracy. Moreover, ExoTST exhibits strong robustness against missing values and noise in exogenous drivers, maintaining consistent performance in real-world situations where these imperfections are common.

时间序列Transformer外生变量预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。