提出TC-BiMamba，实现动态分块训练，兼顾流式与非流式语音识别。
TC-BiMamba: Trans-Chunk bidirectionally within BiMamba for unified streaming and non-streaming ASR
- 创新性地设计跨块双向处理机制，支持动态分块训练
- 训练速度提升1.3倍，内存降低50%，性能更优
- 适合需要灵活延迟设置的语音识别系统
本文研究用于统一流式与非流式自动语音识别（ASR）的双向Mamba（BiMamba）。动态分块训练使单一模型可支持不同延迟设置下的离线与流式解码。现有基于BiMamba的流式方法受限于固定分块大小。当引入动态分块训练时，训练开销显著增加。为此，我们提出跨块双向Mamba（TC-BiMamba），支持动态分块训练下的双向序列建模。相比传统分块处理，TC-BiMamba在离线模式下同时实现1.3倍训练加速、50%训练内存减少，并因捕获双向上下文而提升模型性能。实验表明，其优于U2++，且在更小模型规模下媲美LC-BiMamba。
原文摘要 · Abstract (English)
This work investigates bidirectional Mamba (BiMamba) for unified streaming and non-streaming automatic speech recognition (ASR). Dynamic chunk size training enables a single model for offline decoding and streaming decoding with various latency settings. In contrast, existing BiMamba based streaming method is limited to fixed chunk size decoding. When dynamic chunk size training is applied, training overhead increases substantially. To tackle this issue, we propose the Trans-Chunk BiMamba (TC-BiMamba) for dynamic chunk size training. Trans-Chunk mechanism trains both bidirectional sequences in an offline style with dynamic chunk size. On the one hand, compared to traditional chunk-wise processing, TC-BiMamba simultaneously achieves 1.3 times training speedup, reduces training memory by 50%, and improves model performance since it can capture bidirectional context. On the other hand, experimental results show that TC-BiMamba outperforms U2++ and matches LC-BiMmaba with smaller model size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。