融合Mamba与Transformer，提升多变量时序预测精度与效率
Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics
- 设计MAT模型，结合Mamba长程依赖与Transformer短程捕捉能力
- 在气象数据集上实现更高预测精度、更低内存占用和更好可扩展性
- 适合需要兼顾长短期模式的复杂时序预测任务
长期-短期时序预测对预测未来趋势至关重要。尽管深度学习模型如Transformer在时序预测中取得显著进展，但常难以捕捉长期依赖关系并有效处理稀疏语义特征。状态空间模型Mamba通过选择性输入和并行计算，较好平衡了计算效率与预测准确性。本文分析了Mamba与Transformer的优劣，提出一种融合模型MAT，充分利用两者优势，捕捉多变量时序中的独特长短程依赖与内在演化模式。具体而言，MAT结合了Mamba的长程依赖建模能力和Transformer的短程特征提取能力。在基准气象数据集上的实验表明，MAT在预测精度、可扩展性和内存效率方面均优于现有可比方法。
原文摘要 · Abstract (English)
Long-short range time series forecasting is essential for predicting future trends and patterns over extended periods. While deep learning models such as Transformers have made significant strides in advancing time series forecasting, they often encounter difficulties in capturing long-term dependencies and effectively managing sparse semantic features. The state-space model, Mamba, addresses these issues through its adept handling of selective input and parallel computing, striking a balance between computational efficiency and prediction accuracy. This article examines the advantages and disadvantages of both Mamba and Transformer models, and introduces a combined approach, MAT, which leverages the strengths of each model to capture unique long-short range dependencies and inherent evolutionary patterns in multivariate time series. Specifically, MAT harnesses the long-range dependency capabilities of Mamba and the short-range characteristics of Transformers. Experimental results on benchmark weather datasets demonstrate that MAT outperforms existing comparable methods in terms of prediction accuracy, scalability, and memory efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。