arXiv:2607.10740cs.LGcs.AI2026-07

用多尺度卷积与最优传输优化注意力,提升时序预测精度

Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series

论文配图:Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series
图 1 · 摘自论文原文
  • 通过反向嵌入将变量转为令牌,捕捉跨变量关系
  • 结合多尺度卷积与Sinkhorn算法,有效抑制噪声并平衡信息流
  • 适用于短中长期预测,适合需要高精度建模的时序任务

多变量时间序列(MTS)分析在众多实际应用中至关重要,但如何捕捉多粒度结构模式并合理抑制噪声仍是挑战。本文提出多尺度卷积与最优传输注意力(MSC-OT)架构,通过反向嵌入将各变量映射为令牌,增强跨变量关系建模能力。该方法包含两部分:(1)多尺度卷积增强,对基于反向嵌入的注意力得分矩阵应用多尺度卷积,捕捉由压缩时间表示诱导的变量子空间中的局部结构模式;(2)Sinkhorn最优传输正则化,将注意力计算建模为最优传输问题,通过迭代矩阵缩放确保跨变量间信息流动均衡。此外,采用softmax归一化的可学习权重动态融合基础注意力、卷积增强及OT正则化得分。在ETT、Electricity、Traffic、Solar-Energy和Exchange-Rate等广泛使用的数据集上实验表明,MSC-OT在短时与长时预测任务中均表现优异。消融实验进一步验证了各组件的有效性及其协同增益。

原文摘要 · Abstract (English)

The analysis of Multivariate Time Series (MTS) plays an important role in a lot of real-world practical applications, but it still remains some challenging problem about capturing multi-granularity structural patterns and suppressing noise appropriately. Multi-Scale Convolution with Optimal Transport Attention (MSC-OT) is proposed in this paper. MSC-OT is a useful architecture to optimize the attention mechanism. It combines multi-scale convolution with Sinkhorn optimal transport method based on inverted embedding. The inverted embedding approach embeds each variable as a token and allows the model to capture cross-variate relationships better. MSC-OT consists of two part: (1) Multi-Scale Convolution Enhancement, that applies multi-scale convolutions to attention score matrices based on inverted embedding, capturing local structural patterns in the variate-interaction space induced by compressed temporal representations; (2) Sinkhorn Optimal Transport Regularization, that formulates attention computation as an optimal transport problem and employs iterative matrix scaling to ensure balanced information flow across variates. Adaptive Fusion Strategy utilizes softmax-normalized learnable weights to dynamically combine base attention, convolution-enhanced, and OT-regularized scores. Experiments on widely-used datasets, including ETT, Electricity, Traffic, Solar-Energy, and Exchange-Rate, show that MSC-OT achieves well performance in both short-term and long-term forecasting tasks. Ablation experiments further validate the effectiveness of each proposed component and their synergistic contributions to improving prediction accuracy for multivariate time series forecasting.

时间序列注意力机制多尺度卷积最优传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。