用音节级掩码建模实现无需配对数据的语音识别
Towards Unsupervised Speech Recognition at the Syllable-Level
- 以音节为单位做掩码语言建模，跳过昂贵的音素转换器
- 在LibriSpeech上字符错误率降低40%，并成功适配中文
- 适合低资源语言和非平行多模态学习场景
使用未配对语音与文本训练语音识别器——即无监督语音识别（UASR）——是将语音识别扩展至长尾分布中的低资源语言，并实现从非平行数据中进行多模态学习的关键一步。然而，现有基于音素的方法常依赖于昂贵的音素转写工具（如G2P），且在音素边界模糊的语言上因训练不稳定而难以泛化。本文提出一种基于掩码语言建模的音节级无监督语音识别框架，避免了G2P需求及生成对抗网络方法的不稳定性。该方法在LibriSpeech上实现了最高达40%的字符错误率（CER）相对下降，并有效推广至中文等复杂语言。代码将在论文接受后发布。
原文摘要 · Abstract (English)
Training speech recognizers with unpaired speech and text -- known as unsupervised speech recognition (UASR) -- is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40\% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to Mandarin, a language that has remained particularly difficult for prior methods. Code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。