用混合模型+自监督训练，让小语种语音识别更准
A Practitioner's Guide to Building ASR Models for Low-Resource Languages: A Case Study on Scottish Gaelic
- 结合隐马尔可夫模型与自监督学习，更好利用有限数据
- 在苏格兰盖尔语上比最佳微调版Whisper降低32%错误率
- 适合数据少的低资源语言语音识别研究者参考
针对低资源语言构建语音识别系统的一种有效方法是微调现有的多语言端到端模型。当原始模型在大量语言的数据上训练时，即使目标语言未出现在原始训练数据中，微调也可在少量数据下取得良好效果。这一方法因公开可用的端到端模型而受到推崇，普遍认为能实现顶尖性能。但本文挑战这一观点：我们证明，结合混合HMM与自监督模型的方法，在数据有限时表现更优。该方法通过持续自监督预训练和半监督训练，更充分地利用所有可用语音与文本数据。我们在苏格兰盖尔语上进行基准测试，相比最优微调版Whisper模型，相对词错误率降低32%。
原文摘要 · Abstract (English)
An effective approach to the development of ASR systems for low-resource languages is to fine-tune an existing multilingual end-to-end model. When the original model has been trained on large quantities of data from many languages, fine-tuning can be effective with limited training data, even when the language in question was not present in the original training data. The fine-tuning approach has been encouraged by the availability of public-domain E2E models and is widely believed to lead to state-of-the-art results. This paper, however, challenges that belief. We show that an approach combining hybrid HMMs with self-supervised models can yield substantially better performance with limited training data. This combination allows better utilisation of all available speech and text data through continued self-supervised pre-training and semi-supervised training. We benchmark our approach on Scottish Gaelic, achieving WER reductions of 32% relative over our best fine-tuned Whisper model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。