arXiv:2510.09528cs.CLcs.SD2025-10被引 4

通过遮蔽关键频谱区域,提升语音识别对口音的鲁棒性。

Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Masking

  • 用频谱分类器定位口音特征区域,进行针对性遮蔽。
  • 在英语和波斯语上分别降低15.2%和18.7%的词错误率。
  • 适合做多语言语音识别、低资源语言研究的开发者使用。

基于预训练变换器的语音识别模型虽有显著进展,但仍对口音和方言变化敏感,导致英语和波斯语等语言中词错误率(WER)偏高。为此,本文提出一种口音不变的语音识别框架,将口音与方言分类集成到识别流程中。方法包括训练基于频谱的分类器捕捉口音特异性线索,遮蔽对其预测最具影响力的频谱区域,并利用遮蔽后的频谱图进行数据增强,从而提升语音识别模型对口音变异的鲁棒性。实验在英语和波斯语上进行,针对波斯语,本文构建了一个涵盖多种地区口音的新数据集,首次建立了波斯语口音变化的系统性基准,填补了多语言语音研究中的关键空白,并为未来低资源、语言多样性研究提供基础。基于Whisper模型的实验结果表明,该遮蔽与增强策略在英、波斯语场景下均实现显著的WER下降,验证了方法有效性。代码与数据集已公开。

原文摘要 · Abstract (English)

Pre-trained transformer-based models have significantly advanced automatic speech recognition (ASR), yet they remain sensitive to accent and dialectal variations, resulting in elevated word error rates (WER) in linguistically diverse languages such as English and Persian. To address this challenge, we propose an accent-invariant ASR framework that integrates accent and dialect classification into the recognition pipeline. Our approach involves training a spectrogram-based classifier to capture accent-specific cues, masking the regions most influential to its predictions, and using the masked spectrograms for data augmentation. This enhances the robustness of ASR models against accent variability. We evaluate the method using both English and Persian speech. For Persian, we introduce a newly collected dataset spanning multiple regional accents, establishing the first systematic benchmark for accent variation in Persian ASR that fills a critical gap in multilingual speech research and provides a foundation for future studies on low-resource, linguistically diverse languages. Experimental results with the Whisper model demonstrate that our masking and augmentation strategy yields substantial WER reductions in both English and Persian settings, confirming the effectiveness of the approach. This research advances the development of multilingual ASR systems that are resilient to accent and dialect diversity. Code and dataset are publicly available at: https://github.com/MH-Sameti/Accent_invariant_ASR

语音识别口音鲁棒数据增强多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。