用Mamba模型提升语音分段精度，效果优于传统RNN和注意力模型。
Mamba-based Segmentation Model for Speaker Diarization
- 用Mamba替代传统RNN或注意力机制进行语音分段
- 在三个主流数据集上达到当前最佳性能
- 支持更长局部窗口，提升说话人嵌入可靠性
Mamba是一种新型架构，兼具循环神经网络（RNN）与类注意力机制特性。针对语音分离任务中注意力模型内存开销大、传统RNN表达能力弱的问题，本文提出基于Mamba的分段模型，并与pyannote管道中的先进神经分段方法对比。Mamba更强的处理能力使局部窗口可显著加长，从而提升说话人嵌入提取的可靠性，大幅改善分离效果。实验表明，该模型在三个广泛使用的语音分离数据集上均达到最优表现，是传统RNN与现有注意力模型的更优替代方案。
原文摘要 · Abstract (English)
Mamba is a newly proposed architecture which behaves like a recurrent neural network (RNN) with attention-like capabilities. These properties are promising for speaker diarization, as attention-based models have unsuitable memory requirements for long-form audio, and traditional RNN capabilities are too limited. In this paper, we propose to assess the potential of Mamba for diarization by comparing the state-of-the-art neural segmentation of the pyannote pipeline with our proposed Mamba-based variant. Mamba's stronger processing capabilities allow usage of longer local windows, which significantly improve diarization quality by making the speaker embedding extraction more reliable. We find Mamba to be a superior alternative to both traditional RNN and the tested attention-based model. Our proposed Mamba-based system achieves state-of-the-art performance on three widely used diarization datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。