提出线性复杂度的EiFormer,让大规模时序预测更快更准。
Towards Efficient Large Scale Spatial-Temporal Time Series Forecasting via Improved Inverted Transformers
- 重构注意力机制,消除冗余计算,实现线性时间复杂度
- 在LargeST和私有数据集上均显著提升预测精度与效率
- 适合需要处理海量同步时序数据的工业级系统
大规模时序预测面临三大挑战:新实体出现、旧实体消失以及实体数量庞大。尽管最近提出的Inverted Transformer(iTransformer)能有效处理可变实体,但其在大规模场景中受限于实体数N的二次方时间与空间复杂度(O(N²))。本文提出EiFormer,一种改进的倒置变换器架构,在保持iTransformer自适应能力的同时,将复杂度降至线性(O(N))。核心创新在于重构注意力机制以消除冗余计算,同时引入随机投影机制,既提升效率又增强特征表示,从而改善预测精度。在公开的LargeST基准数据集及一个私有大规模时序数据集上的实验表明,EiFormer在计算效率和预测准确率上均显著优于现有方法。该方法使基于Transformer的大规模时序预测在工业应用中具备实际部署可行性。
原文摘要 · Abstract (English)
Time series forecasting at scale presents significant challenges for modern prediction systems, particularly when dealing with large sets of synchronized series, such as in a global payment network. In such systems, three key challenges must be overcome for accurate and scalable predictions: 1) emergence of new entities, 2) disappearance of existing entities, and 3) the large number of entities present in the data. The recently proposed Inverted Transformer (iTransformer) architecture has shown promising results by effectively handling variable entities. However, its practical application in large-scale settings is limited by quadratic time and space complexity ($O(N^2)$) with respect to the number of entities $N$. In this paper, we introduce EiFormer, an improved inverted transformer architecture that maintains the adaptive capabilities of iTransformer while reducing computational complexity to linear scale ($O(N)$). Our key innovation lies in restructuring the attention mechanism to eliminate redundant computations without sacrificing model expressiveness. Additionally, we incorporate a random projection mechanism that not only enhances efficiency but also improves prediction accuracy through better feature representation. Extensive experiments on the public LargeST benchmark dataset and a proprietary large-scale time series dataset demonstrate that EiFormer significantly outperforms existing methods in both computational efficiency and forecasting accuracy. Our approach enables practical deployment of transformer-based forecasting in industrial applications where handling time series at scale is essential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。