arXiv:2512.23824cs.LG2025-12被引 3

提出多尺度状态空间模型,提升长序列建模效率与准确性

MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling

  • 通过多分辨率处理捕捉不同尺度的序列动态
  • 在长程依赖任务中显著优于现有SSM模型
  • 适合需要高效建模复杂结构的长序列场景

状态空间模型(SSMs)作为计算成本较低的注意力模型替代方案,近年来受到关注。它们基于线性递推实现时间信息融合,支持快速推理、可并行训练并控制递推稳定性。然而传统SSM有效记忆能力有限,需增大状态尺寸以提升召回率;且难以捕捉多尺度依赖,影响对时间序列、图像和自然语言中复杂结构的建模。本文提出多尺度状态空间模型(MS-SSM),通过在多个分辨率上表示序列动态,并为各分辨率设计专用的状态空间演化机制,同时捕捉高频细节与全局趋势,从而提升记忆效率与长程建模能力。进一步引入输入相关尺度混合器,实现跨分辨率动态信息融合。大量实验表明,在长程任务、层次推理、时间序列分类和图像识别等基准上,MS-SSM持续超越先前基于SSM的模型,验证了多分辨率处理在状态空间架构中的优势。

原文摘要 · Abstract (English)

State-space models (SSMs) have recently attention as an efficient alternative to computationally expensive attention-based models for sequence modeling. They rely on linear recurrences to integrate information over time, enabling fast inference, parallelizable training, and control over recurrence stability. However, traditional SSMs often suffer from limited effective memory, requiring larger state sizes for improved recall. Moreover, existing SSMs struggle to capture multi-scale dependencies, which are essential for modeling complex structures in time series, images, and natural language. This paper introduces a multi-scale SSM framework that addresses these limitations by representing sequence dynamics across multiple resolution and processing each resolution with specialized state-space dynamics. By capturing both fine-grained, high-frequency patterns and coarse, global trends, MS-SSM enhances memory efficiency and long-range modeling. We further introduce an input-dependent scale-mixer, enabling dynamic information fusion across resolutions. The proposed approach significantly improves sequence modeling, particularly in long-range and hierarchical tasks, while maintaining computational efficiency. Extensive experiments on benchmarks, including Long Range Arena, hierarchical reasoning, time series classification, and image recognition, demonstrate that MS-SSM consistently outperforms prior SSM-based models, highlighting the benefits of multi-resolution processing in state-space architectures.

状态空间模型长序列建模多尺度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。