用数据增强提升苏丹方言语音识别，仅用少量数据达到高精度。
Doing More with Less: Data Augmentation for Sudanese Dialect Automatic Speech Recognition
- 结合自训练与语音合成生成新数据，提升模型性能
- 最佳模型在测试集上错词率降至51.6%，优于零样本模型30个百分点
- 开源工具链，为低资源语言提供可复现的开发路径
尽管已有大量面向现代标准阿拉伯语（MSA）和方言阿拉伯语（DA）的自动语音识别（ASR）系统，但针对特定方言、尤其是苏丹方言等低资源阿拉伯语方言的研究仍很少。本文对OpenAI Whisper模型的微调策略进行了全面的数据增强研究，并建立了首个苏丹方言的基准评估体系。探索了两种增强方法：(1) 基于未标注语音生成伪标签的自训练；(2) 使用Klaam TTS系统生成合成语音的文本到语音增强。最优模型——在28.4小时数据上联合使用自训练与TTS增强的Whisper-Medium——在测试集上达到57.1%的词错误率（WER），在跨域保留集上达51.6%，显著优于零样本多语言Whisper（78.8% WER）及专用于MSA的阿拉伯语模型（73.8%-123% WER）。所有实验均基于低成本资源（Kaggle免费层与Lightning.ai试用版），证明了策略性数据增强可在资源受限条件下有效提升低资源方言识别性能，为发展低资源阿拉伯语及其他边缘语言的语音识别提供了实用路线。模型、评估基准与可复现训练流程已公开发布，以促进后续研究。
原文摘要 · Abstract (English)
Although many Automatic Speech Recognition (ASR) systems have been developed for Modern Standard Arabic (MSA) and Dialectal Arabic (DA), few studies have focused on dialect-specific implementations, particularly for low-resource Arabic dialects such as Sudanese. This paper presents a comprehensive study of data augmentation techniques for fine-tuning OpenAI Whisper models and establishes the first benchmark for the Sudanese dialect. Two augmentation strategies are investigated: (1) self-training with pseudo-labels generated from unlabeled speech, and (2) TTS-based augmentation using synthetic speech from the Klaam TTS system. The best-performing model, Whisper-Medium fine-tuned with combined self-training and TTS augmentation (28.4 hours), achieves a Word Error Rate (WER) of 57.1% on the evaluation set and 51.6% on an out-of-domain holdout set substantially outperforming zero-shot multilingual Whisper (78.8% WER) and MSA-specialized Arabic models (73.8-123% WER). All experiments used low-cost resources (Kaggle free tier and Lightning.ai trial), demonstrating that strategic data augmentation can overcome resource limitations for low-resource dialects and provide a practical roadmap for developing ASR systems for low-resource Arabic dialects and other marginalized language varieties. The models, evaluation benchmarks, and reproducible training pipelines are publicly released to facilitate future research on low-resource Arabic ASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。