用少量标注数据+伪标签继续预训练，让语音模型在斯瓦希里语上达到顶尖水平。
Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
- 用未标注音频和少量标注数据做伪标签继续预训练
- 仅2万标注样本就实现3.24%词错误率
- 方法可复现，适合其他低资源语言
本文研究将wav2vec2-bert-2.0通过持续预训练（CPT）适配至斯瓦希里语自动语音识别（ASR）。方法结合未标注音频与有限标注数据，先进行伪标签持续预训练，再进行监督微调。使用20,000个标注样本，在Common Voice Swahili数据集上取得3.24%的词错误率（WER），相比基线实现82%相对提升。该结果优于此前最佳学术系统（XLS-R，8.3% WER）61%相对改进。论文提供了具体的数据需求和可复现的方法流程，适用于其他低资源语言。
原文摘要 · Abstract (English)
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。