通过多阶段微调,用少量数据提升小语种语音识别准确率。
Multistage Fine-tuning Strategies for Automatic Speech Recognition in Low-resource Languages
- 先在相关语言上训练中间模型,再逐步迁移到目标语言。
- 在马拉萨尔语上实现51.9%的词错误率,比直接微调低4.5个百分点。
- 适合资源匮乏但有语言相似性的低资源语种语音系统开发。
本文提出一种新型多阶段微调策略,旨在利用OpenAI Whisper模型提升低资源语言的自动语音识别(ASR)性能。针对印度西高止山区约一万人使用的德拉威语系马拉萨尔语,该语言缺乏原生文字和数字语音资源,技术干预困难。研究团队与Wycliffe India及本地社区合作,构建了以泰米尔文字转录的马拉萨尔语语音语料库。方法上,先基于数据丰富的泰米尔语建立中间ASR模型,再将其在马拉萨尔语数据上微调,实现更有效的迁移。相比直接在马拉萨尔语上微调,该策略将词错误率(WER)降至51.9%,降低4.5个百分点;进一步通过后处理去除标点符号,将WER降至47.3%,缓解了格式不一致带来的评估偏差。结果表明,结合语言相似性与多阶段微调及针对性后处理,是低资源语言语音识别可扩展的有效路径。
原文摘要 · Abstract (English)
This paper presents a novel multistage fine-tuning strategy designed to enhance automatic speech recognition (ASR) performance in low-resource languages using OpenAI's Whisper model. In this approach we aim to build ASR model for languages with limited digital resources by sequentially adapting the model across linguistically similar languages. We experimented this on the Malasar language, a Dravidian language spoken by approximately ten thousand people in the Western Ghats of South India. Malasar language faces critical challenges for technological intervention due to its lack of a native script and absence of digital or spoken data resources. Working in collaboration with Wycliffe India and Malasar community members, we created a spoken Malasar corpus paired with transcription in Tamil script, a closely related major language. In our approach to build ASR model for Malasar, we first build an intermediate Tamil ASR, leveraging higher data availability for Tamil annotated speech. This intermediate model is subsequently fine-tuned on Malasar data, allowing for more effective ASR adaptation despite limited resources. The multistage fine-tuning strategy demonstrated significant improvements over direct fine-tuning on Malasar data alone, achieving a word error rate (WER) of 51.9%, which is 4.5% absolute reduction when compared to the direct fine-tuning method. Further a WER reduction to 47.3% was achieved through punctuation removal in post-processing, which addresses formatting inconsistencies that impact evaluation. Our results underscore the effectiveness of sequential multistage fine-tuning combined with targeted post-processing as a scalable strategy for ASR system development in low-resource languages, especially where linguistic similarities can be leveraged to bridge gaps in training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。