改进Transformer模型，更好捕捉时间序列的长期依赖和历史累积信息。
SCFormer: Structured Channel-wise Transformer with Cumulative Historical State for Multivariate Time Series Forecasting
- 引入时序约束到所有线性变换，增强时间建模能力
- 使用HiPPO处理历史序列，突破回看窗口限制
- 在多个真实数据集上优于主流基线方法
Transformer在多变量时间序列预测中表现优异，主要依赖通道级自注意力机制。然而，该方法在计算时间特征时缺乏时序约束，且未能有效利用累积历史序列。为解决这些问题，我们提出结构化通道级Transformer与累积历史状态（SCFormer）。SCFormer对查询、键、值矩阵及变换层中的全连接层均引入时序约束，并采用高阶多项式投影算子（HiPPO）处理累积历史序列，使模型可在预测时融入超出回看窗口的历史信息。在多个真实世界数据集上的大量实验表明，SCFormer显著优于主流基线模型，验证了其在提升时间序列预测性能方面的有效性。代码已公开于https://github.com/ShiweiGuo1995/SCFormer。
原文摘要 · Abstract (English)
The Transformer model has shown strong performance in multivariate time series forecasting by leveraging channel-wise self-attention. However, this approach lacks temporal constraints when computing temporal features and does not utilize cumulative historical series effectively.To address these limitations, we propose the Structured Channel-wise Transformer with Cumulative Historical state (SCFormer). SCFormer introduces temporal constraints to all linear transformations, including the query, key, and value matrices, as well as the fully connected layers within the Transformer. Additionally, SCFormer employs High-order Polynomial Projection Operators (HiPPO) to deal with cumulative historical time series, allowing the model to incorporate information beyond the look-back window during prediction. Extensive experiments on multiple real-world datasets demonstrate that SCFormer significantly outperforms mainstream baselines, highlighting its effectiveness in enhancing time series forecasting. The code is publicly available at https://github.com/ShiweiGuo1995/SCFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。