arXiv:2502.20571cs.LGcs.AI2025-02KDD被引 3

PFformer无需位置编码,提升极端多变量时序预测精度。

PFformer: A Position-Free Transformer Variant for Extreme-Adaptive Multivariate Time Series Forecasting

  • 用无位置约束的嵌入策略捕捉变量间复杂依赖关系
  • 在四个挑战性数据集上提升20%至60%的预测准确率
  • 适合处理极端波动、稀疏事件频发的真实场景

多变量时间序列(MTS)预测在气象、能源和金融等领域至关重要。然而,尽管深度学习取得进展,传统Transformer模型因单一令牌嵌入方式削弱了关键变量间关系,并难以有效捕捉变量间的复杂依赖,尤其在罕见或极端事件频发的数据集中。这些事件导致显著不平衡与高偏度,增加预测难度。本文提出PFformer,一种专为单目标多变量时间序列预测设计的位置无关Transformer模型,适用于极端可变数据集。该模型融合两种新型嵌入策略:增强特征嵌入(EFE)与自编码器嵌入(AEE)。EFE通过将相关序列子集映射到高维空间,无需位置约束即可有效编码变量间依赖,增强编码器功能。实验在四个挑战性数据集上进行,涵盖两个关键预测场景:提前3天的长序列预测,以及每4小时滚动预测,以模拟水资源管理中的实时决策过程。结果表明,相比现有最优模型,PFformer实现20%至60%的性能提升。

原文摘要 · Abstract (English)

Multivariate time series (MTS) forecasting is vital in fields like weather, energy, and finance. However, despite deep learning advancements, traditional Transformer-based models often diminish the effect of crucial inter-variable relationships by singular token embedding and struggle to effectively capture complex dependencies among variables, especially in datasets with rare or extreme events. These events create significant imbalances and lead to high skewness, complicating accurate prediction efforts. This study introduces PFformer, a position-free Transformer-based model designed for single-target MTS forecasting, specifically for challenging datasets characterized by extreme variability. PFformer integrates two novel embedding strategies: Enhanced Feature-based Embedding (EFE) and Auto-Encoder-based Embedding (AEE). EFE effectively encodes inter-variable dependencies by mapping related sequence subsets to high-dimensional spaces without positional constraints, enhancing the encoder's functionality. PFformer shows superior forecasting accuracy without the traditional limitations of positional encoding in MTS modeling. We evaluated PFformer across four challenging datasets, focusing on two key forecasting scenarios: long sequence prediction for 3 days ahead and rolling predictions every four hours to reflect real-time decision-making processes in water management. PFformer demonstrated remarkable improvements, from 20% to 60%, compared with state-of-the-art models.

时间序列Transformer多变量极端预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。