给Transformer加'嵌入护甲'，提升多变量时间序列预测性能
EMAformer: Enhancing Transformer through Embedding Armor for Time Series Forecasting
- 引入全局稳定、相位敏感、跨轴特异性三类先验，增强通道间关系建模
- 在12个真实数据集上平均降低2.73%的MSE误差和5.15%的MAE误差
- 适合需要高精度多变量时间序列预测的工业与金融场景
多变量时间序列预测在众多领域至关重要。尽管Transformer架构已取得显著进展，但iTransformer仍落后于最新的MLP模型。我们归因于通道间关系的不稳定性。为此，提出EMAformer，一种通过辅助嵌入套件增强Transformer的简单而有效的方法，类似于为其提供护甲以强化能力。通过引入三个关键归纳偏置——全局稳定性、相位敏感性、跨轴特异性，EMAformer释放了Transformer架构的更大潜力，在12个真实世界基准上达到最先进性能，平均将预测误差降低2.73%（MSE）和5.15%（MAE）。这显著提升了基于Transformer的方法在多变量时间序列预测中的实际应用价值。代码已公开于https://github.com/PlanckChang/EMAformer。
原文摘要 · Abstract (English)
Multivariate time series forecasting is crucial across a wide range of domains. While presenting notable progress for the Transformer architecture, iTransformer still lags behind the latest MLP-based models. We attribute this performance gap to unstable inter-channel relationships. To bridge this gap, we propose EMAformer, a simple yet effective model that enhances the Transformer with an auxiliary embedding suite, akin to armor that reinforces its ability. By introducing three key inductive biases, i.e., \textit{global stability}, \textit{phase sensitivity}, and \textit{cross-axis specificity}, EMAformer unlocks the further potential of the Transformer architecture, achieving state-of-the-art performance on 12 real-world benchmarks and reducing forecasting errors by an average of 2.73\% in MSE and 5.15\% in MAE. This significantly advances the practical applicability of Transformer-based approaches for multivariate time series forecasting. The code is available on https://github.com/PlanckChang/EMAformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。