用低秩神经微分方程增强Mamba,高效建模长时序数据。
MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs
- 结合低秩神经微分方程与改进Mamba架构,捕捉时间动态。
- 在多个基准上实现更高精度与更低计算开销。
- 适合处理长序列、不规则采样时序数据的场景。
时序预测在金融、医疗、能源和环境建模等领域至关重要。然而,现有方法往往难以在效率、可扩展性与准确性之间取得平衡,尤其面对长程依赖和不规则采样数据时。为此,我们提出MODE框架,将低秩神经常微分方程(Neural ODEs)与增强型Mamba架构融合。输入序列首先通过线性标记化层处理,再经多个配备增强Mamba层的Mamba编码器块,每层包含因果卷积、SiLU激活及低秩神经ODE增强,以高效捕捉时间动态。该低秩形式降低计算开销,同时保持表达能力。此外,受伪微分方程启发的分段选择性扫描机制,自适应聚焦关键子序列,提升可扩展性与长序列建模能力。大量实验表明,MODE在预测精度与计算效率上均优于现有基线。主要贡献包括:(1) 统一高效的长期时序建模架构;(2) 将Mamba的选择性扫描与低秩神经ODE结合,增强时间表征;(3) 通过低秩近似与动态选择性扫描显著提升效率与可扩展性。
原文摘要 · Abstract (English)
Time series prediction plays a pivotal role across diverse domains such as finance, healthcare, energy systems, and environmental modeling. However, existing approaches often struggle to balance efficiency, scalability, and accuracy, particularly when handling long-range dependencies and irregularly sampled data. To address these challenges, we propose MODE, a unified framework that integrates Low-Rank Neural Ordinary Differential Equations (Neural ODEs) with an Enhanced Mamba architecture. As illustrated in our framework, the input sequence is first transformed by a Linear Tokenization Layer and then processed through multiple Mamba Encoder blocks, each equipped with an Enhanced Mamba Layer that employs Causal Convolution, SiLU activation, and a Low-Rank Neural ODE enhancement to efficiently capture temporal dynamics. This low-rank formulation reduces computational overhead while maintaining expressive power. Furthermore, a segmented selective scanning mechanism, inspired by pseudo-ODE dynamics, adaptively focuses on salient subsequences to improve scalability and long-range sequence modeling. Extensive experiments on benchmark datasets demonstrate that MODE surpasses existing baselines in both predictive accuracy and computational efficiency. Overall, our contributions include: (1) a unified and efficient architecture for long-term time series modeling, (2) integration of Mamba's selective scanning with low-rank Neural ODEs for enhanced temporal representation, and (3) substantial improvements in efficiency and scalability enabled by low-rank approximation and dynamic selective scanning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。