用音节级掩码建模实现无需标注的语音识别，提升低资源语言性能。
Unsupervised Speech Recognition at the Syllable Level
- 基于音节级掩码语言模型，避开发音转换器和不稳定的生成对抗训练
- 在LibriSpeech上字符错误率降低40%，对低资源语言泛化能力更强
- 适合研究低资源语音识别与无监督多模态学习的学者使用
在无配对语音与文本数据的情况下训练语音识别器——即无监督语音识别（UASR）——是扩展语音识别至长尾分布中的低资源语言，并从非平行数据中实现多模态学习的关键一步。然而，现有基于音素的方法通常依赖昂贵的音素转写工具（如G2P），且在音素边界模糊的语言上因训练不稳定而表现不佳。本文提出一种基于掩码语言建模的音节级无监督语音识别框架，避免了G2P依赖及生成对抗方法的不稳定性。该方法在LibriSpeech上实现了最高达40%的字符错误率相对降低，并有效推广至以往方法难以处理的低资源语言。代码已公开。
原文摘要 · Abstract (English)
Training speech recognizers with unpaired speech and text -- known as unsupervised speech recognition (UASR) -- is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40\% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to low-resource languages that have remained particularly difficult for prior methods. Code is publicly available\footnote{https://github.com/cactuswiththoughts/SylCipher}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。