用状态空间模型提升语音识别长序列建模能力
Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models
- 在Transformer架构中引入选择性状态空间机制
- 处理长语音时计算复杂度接近线性增长
- 适合需要长上下文的语音识别任务
当前基于Transformer的自动语音识别系统因二次复杂度难以处理长语音序列。选择性状态空间模型（如Mamba）在自然语言和计算机视觉任务中表现优异，但在语音技术领域研究较少。本文提出Speech-Mamba，将选择性状态空间建模融入Transformer神经网络架构，结合状态空间模型对长序列的表征能力与Transformer的低层特征提取优势。该方法在保持高表达力的同时，使模型在序列长度增加时计算开销近似线性增长，显著提升了长程依赖建模能力。
原文摘要 · Abstract (English)
Current automatic speech recognition systems struggle with modeling long speech sequences due to high quadratic complexity of Transformer-based models. Selective state space models such as Mamba has performed well on long-sequence modeling in natural language processing and computer vision tasks. However, research endeavors in speech technology tasks has been under-explored. We propose Speech-Mamba, which incorporates selective state space modeling in Transformer neural architectures. Long sequence representations with selective state space models in Speech-Mamba is complemented with lower-level representations from Transformer-based modeling. Speech-mamba achieves better capacity to model long-range dependencies, as it scales near-linearly with sequence length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。