提出有序鲁棒Mamba模型，解决时间序列通道顺序敏感问题
Sequential Order-Robust Mamba for Time Series Forecasting
- 通过反序数据嵌入差异正则化，提升对通道顺序的鲁棒性
- 移除1D卷积并引入通道相关性建模预训练任务
- 适合关注时间序列通道依赖建模的研究者与应用者
Mamba最近成为Transformer的有力替代，可近似线性复杂度处理序列数据。然而，时间序列数据中通道通常无固定顺序，现有研究在捕捉通道依赖（CD）时引入了顺序偏差。为此，我们提出SOR-Mamba，通过两项改进：1）设计正则化策略，最小化反序通道输入产生的嵌入向量差异，增强对通道顺序的鲁棒性；2）移除原用于捕捉局部信息的1D卷积。此外，提出通道相关性建模（CCM）作为预训练任务，旨在将数据空间中的通道相关性保留到潜在空间，以强化对通道依赖的建模能力。大量实验表明，该方法在标准和迁移学习场景下均表现优异。代码已公开于https://github.com/seunghan96/SOR-Mamba。
原文摘要 · Abstract (English)
Mamba has recently emerged as a promising alternative to Transformers, offering near-linear complexity in processing sequential data. However, while channels in time series (TS) data have no specific order in general, recent studies have adopted Mamba to capture channel dependencies (CD) in TS, introducing a sequential order bias. To address this issue, we propose SOR-Mamba, a TS forecasting method that 1) incorporates a regularization strategy to minimize the discrepancy between two embedding vectors generated from data with reversed channel orders, thereby enhancing robustness to channel order, and 2) eliminates the 1D-convolution originally designed to capture local information in sequential data. Furthermore, we introduce channel correlation modeling (CCM), a pretraining task aimed at preserving correlations between channels from the data space to the latent space in order to enhance the ability to capture CD. Extensive experiments demonstrate the efficacy of the proposed method across standard and transfer learning scenarios. Code is available at https://github.com/seunghan96/SOR-Mamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。