arXiv:2505.03697eess.AS2025-05被引 3

通过按严重程度混合语音,提升腭裂语音的识别公平性。

Improving ASR Fairness for Cleft Lip and Palate Speech: A Study on Severity-Aware Data Mixing

  • 按腭裂严重程度分级混合正常与异常语音,优化模型训练
  • 在印地语和英语数据集上,错误率从37.58%降至25.47%
  • 适合关注语音识别公平性与残障群体技术包容的研究者

腭裂患者语音常因结构异常导致鼻音过重(有时伴气声),形成共振峰偏移,降低自动语音识别(ASR)性能与公平性。主流ASR系统对非典型语音表现较差,本文实证评估了谷歌语音转录等服务在腭裂语音上的公平性下降问题。为一致量化公平性,提出简单公平性分数(FS),权衡整体错误率与组间差异。尽管存在共振峰干扰,轻中度腭裂语音仍部分保留与正常语音的谱时对齐特性,支持采用混合策略提升性能。在AIISH(卡纳达语)和NMCPC(英语)数据集上,系统评估不同严重程度下腭裂与正常语音的严重程度感知混合策略。结果显示,该策略显著提升英语(NMCPC)和卡纳达语(AIISH)数据集的公平性。其中,GMM-HMM模型在AIISH上词错误率(WER)从37.58%降至25.47%,Whisper模型在NMCPC上从35.74%降至21.72%。

原文摘要 · Abstract (English)

Speech produced by individuals with cleft lip and palate (CLP) is often hypernasal (and sometimes breathy) due to structural anomalies, yielding shifts in formant structure that degrade automatic speech recognition (ASR) performance and fairness. Building on evidence that mainstream ASR systems underperform on atypical and disordered speech, we posit that widely used services (e.g., Google Speech-to-Text) exhibit reduced fairness for CLP speech, and we evaluate this claim empirically. To quantify fairness consistently, we introduce a simple fairness score (FS) that trades off overall error and between-group disparity. Despite formant disruptions, mild and moderate CLP speech retains partial spectro-temporal alignment with typical speech, motivating the use of mixing strategies to improve fairness. We systematically investigated severity-aware mixing of CLP and normal speech at different severity levels and assessed its effect on fairness. Three ASR models GMM-HMM, Whisper, and XLSR were evaluated on the AIISH (Kannada language) and NMCPC (English language) datasets. A mixing strategy that leverages severity-aware mixing of CLP and normal speech improves fairness on both English (NMCPC) and Kannada (AIISH) corpora. Notably, the word error rate (WER) decreased from 37.58% to 25.47% (GMM-HMM, AIISH) and from 35.74% to 21.72% (Whisper, NMCPC).

语音识别公平性腭裂语音数据混合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。