arXiv:2411.01419cs.LGcs.AI2024-11被引 5

用分段注意力与参数共享,让时序预测更高效准确

PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting

  • 通过参数共享减少模型参数量,提升效率
  • 引入分段注意力机制,更好捕捉时空局部与全局依赖
  • 在多个基准数据集上优于主流方法,适合大规模时序预测

时序预测在多个领域中仍面临挑战,常因高维数据和长程依赖而复杂。本文提出一种新型Transformer架构PSformer,融合参数共享(PS)与时空分段注意力(SegAtt)。定义时序片段为不同变量相同位置序列块的拼接。通过参数共享机制显著降低训练参数量,提升模型效率与可扩展性。SegAtt通过在片段上计算注意力,增强对局部时空依赖的捕捉能力,并通过跨片段信息融合提升全局表征。二者结合大幅提高预测性能。大量实验表明,PSformer在准确率和可扩展性方面均优于主流基线与其它Transformer方法,成为精准且高效的时序预测工具。

原文摘要 · Abstract (English)

Time series forecasting remains a critical challenge across various domains, often complicated by high-dimensional data and long-term dependencies. This paper presents a novel transformer architecture for time series forecasting, incorporating two key innovations: parameter sharing (PS) and Spatial-Temporal Segment Attention (SegAtt). We also define the time series segment as the concatenation of sequence patches from the same positions across different variables. The proposed model, PSformer, reduces the number of training parameters through the parameter sharing mechanism, thereby improving model efficiency and scalability. The introduction of SegAtt could enhance the capability of capturing local spatio-temporal dependencies by computing attention over the segments, and improve global representation by integrating information across segments. The combination of parameter sharing and SegAtt significantly improves the forecasting performance. Extensive experiments on benchmark datasets demonstrate that PSformer outperforms popular baselines and other transformer-based approaches in terms of accuracy and scalability, establishing itself as an accurate and scalable tool for time series forecasting.

时序预测Transformer参数高效注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。