arXiv:2607.01502cs.CL2026-07

评估Mamba在南非七种语言语音识别中的表现,发现其高效且可跨语种迁移。

From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages

论文配图:From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
图 1 · 摘自论文原文
  • 用Mamba模型处理多语种语音识别,通过语言与语系嵌入提升泛化能力。
  • 相比Conformer,Mamba在相同参数下更快训练且资源更少,准确率相当。
  • 语言嵌入虽不反映语言类型相似性,但能增强跨数据集鲁棒性,适合低资源场景。

近期自动语音识别(ASR)研究探索了不同序列模型,包括基于Conformer的模型和新型状态空间模型Mamba。尽管已有工作评估这些架构在多种语言中的表现,但其在非洲语言中的有效性仍待深入。本文评估了Mamba在七种南非语言上的ASR性能。单语实验中,每种语言使用50小时语音训练,对比同规模参数的Conformer基线。Mamba达到相近识别准确率,同时计算资源更少、训练更快。进一步分析发现,两种模型在长语音上的泛化能力均较弱。随后考察多语种训练,基线为合并所有语言数据;在此基础上测试三种扩展:加入语言及语系嵌入作为下采样声学表示的偏置,以及结合CTC ASR目标与语言识别(LID)头的多任务学习。结果表明,多语训练持续优于单语训练;但显式引入语言信息未提升域内性能,却增强了跨语料库鲁棒性。在低资源设置(每语言5小时和10小时)中进行消融实验,发现语言嵌入带来增益,移除或修改它们会损害性能。最后分析嵌入发现,其不反映语言类型学相似性,而是作为特定任务的控制向量。

原文摘要 · Abstract (English)

Recent advances in automatic speech recognition (ASR) have explored different sequence models, including Conformer-based models and newer state space models such as Mamba. Although prior work has evaluated these architectures in multiple languages, their effectiveness in African languages remains underexplored. In this work, we evaluate Mamba for ASR on seven South African languages. In monolingual experiments, each model is trained on 50 hours of speech per language, and we compare Mamba to a Conformer baseline of similar parameter scale. Mamba achieves similar recognition accuracy to Conformer while using fewer computational resources and training faster. We further evaluate generalization in this setting and find that both models struggle to generalize to speech that is much longer than what they were trained on. We then study multilingual ASR using Mamba models, where the baseline is pooling all languages together. On top of this, we tested three extensions: training with language-family information by adding both language and language-family embeddings as biases to the downsampled acoustic representations, and multitask learning with a CTC ASR objective and a language identification (LID) head. We find that multilingual training consistently improves performance over monolingual training. However, adding explicit language information does not improve in-domain performance but does improve cross-corpus robustness. We conducted ablation studies in low-resource multilingual settings using 5-hour and 10-hour per-language training data, where we observed gains from using language embeddings and further demonstrated that removing or altering them hurt model performance. Lastly, we analysed these embeddings and find that they do not capture linguistic similarity in a typological sense, but instead act as task-specific control vectors.

语音识别多语种Mamba低资源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。