用多尺度混合专家模型提升长期时空序列预测精度
STM3: Mixture of Multiscale Mamba for Long-Term Spatio-Temporal Time-Series Prediction
- 设计多尺度Mamba结合解耦专家混合框架,高效捕捉多粒度时间特征
- 在PEMSD8数据集上比第二名模型降低7.1% MAE、8.5% RMSE、15.9% MAPE
- 适合需要高精度长期预测的交通、气象等时空数据任务
近期时空序列预测发展迅速,但现有深度学习方法难以高效学习复杂的长期时空依赖。长期依赖带来两个新挑战:1)长时序天然包含多尺度信息,难高效提取;2)不同节点的多尺度信息高度相关,难建模。为此,我们提出时空多尺度混合马尔可夫(STM3)。STM3将多尺度Mamba架构融入新型解耦专家混合(DMoE)框架,高效捕捉多样化多尺度信息,同时利用自适应图因果网络建模复杂空间依赖。为确保稳健表征学习,引入稳定路由策略与因果对比学习策略,协同分层信息聚合以保证尺度可区分性。理论上证明了STM3具备更优路由平滑性,并保障每个专家的模式解耦。在10个跨领域的实际基准上广泛实验,表现优异,达到当前最优水平。尤其在PEMSD8数据集上,显著优于次优模型:MAE降低7.1%,RMSE降低8.5%,MAPE降低15.9%。代码已开源。
原文摘要 · Abstract (English)
Recently, spatio-temporal time-series prediction has developed rapidly, yet existing deep learning methods struggle with learning complex long-term spatio-temporal dependencies efficiently. The long-term spatio-temporal dependency learning brings two new challenges: 1) The long-term temporal sequence naturally includes multiscale information, which is hard to extract efficiently; 2) The multiscale temporal information from different nodes is highly correlated and hard to model. To address these challenges, we propose Spatio-Temporal Mixture of Multiscale Mamba (STM3). STM3 integrates a Multiscale Mamba architecture within a novel Disentangled Mixture-of-Experts (DMoE) framework to capture diverse multiscale information efficiently, while utilizing an adaptive graph causal network to model complex spatial dependencies. To ensure robust representation learning, we introduce a stable routing strategy and a causal contrastive learning strategy, which work in tandem with hierarchical information aggregation to guarantee scale distinguishability. We theoretically prove that STM3 achieves superior routing smoothness and guarantees pattern disentanglement for each expert. Extensive experiments on 10 real-world benchmarks across domains demonstrate STM3's superior performance, achieving state-of-the-art results in long-term spatio-temporal time-series prediction. Notably, on the PEMSD8 dataset, it achieves significant improvements, surpassing the second-best model by 7.1% in MAE, 8.5% in RMSE, and 15.9% in MAPE. Code is available at https://github.com/IfReasonable/STM3_KDD26.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。