针对唇腭裂语音识别难题，提出可部署于边缘设备的公平性增强识别框架。
Fairness Evaluation of Edge-AI Implementation for Cleft Lip and Palate Speech ASR
- 基于Whisper-small模型，融合不同严重程度的唇腭裂语音数据进行微调。
- 最佳模型将整体词错误率降至22.72%，音素错误率降至18.44%。
- 可在Jetson设备上实时运行，适合无网络环境下的语音交互场景。
唇腭裂（CLP）人群的语音自动识别（ASR）因病理语音数据有限及个体间发音差异大而面临挑战，尤其在云服务不可用时更影响语音人机交互的可及性。本文研究了一种面向严重程度感知且可部署于边缘的ASR框架，使用Whisper-small模型，在包含轻度、中度、重度及仅唇腭裂语音的数据组合下进行微调，评估不同严重程度数据对识别性能与公平性的影响。预训练模型的总体词错误率（WER）和音素错误率（PER）分别为62.46%和52.72%。严重程度感知微调显著提升性能，最佳模型使总体WER降至22.72%，PER降至18.44%。更广泛涵盖严重程度的数据配置实现了准确率与跨群体表现一致性之间的最优平衡。在NVIDIA Jetson平台部署验证了实时推理能力，实时因子为0.167–0.171，峰值显存占用约566 MB。结果表明，训练阶段引入严重程度多样性可显著提升唇腭裂语音识别效果，并缩小不同严重程度间的性能差距。该方法支持低延迟、无需互联网的语音交互，助力唇腭裂人群实现更普惠的语音人机交互。
原文摘要 · Abstract (English)
Automatic speech recognition (ASR) remains challenging for individuals with cleft lip and palate (CLP) because of limited pathological speech data and large variations in speech characteristics across speakers and severity levels. These recognition difficulties can reduce the accessibility of voice-based human-computer interaction, particularly when cloud-based ASR services are unavailable or unreliable. This work investigates a severity-aware and edge-deployable ASR framework for improving recognition of CLP speech using Whisper-small. The model was fine-tuned using different combinations of normal and CLP speech representing mild, moderate, and severe conditions, together with a CLP-only training configuration, to examine how the inclusion of different severity levels influences recognition performance and fairness across speakers. The pretrained model produced pooled word error rate (WER) and phoneme error rate (PER) values of 62.46% and 52.72%, respectively. Severity-aware fine-tuning substantially improved performance, reducing the best pooled WER to 22.72% and the best pooled PER to 18.44%. Training with a broader representation of CLP severity levels also provided the best overall balance between recognition accuracy and performance consistency across severity groups. Deployment on an NVIDIA Jetson platform demonstrated real-time inference for all fine-tuned models, with real-time factors of 0.167-0.171 and peak GPU memory usage of approximately 566 MB. The results demonstrate that incorporating severity diversity during ASR adaptation can substantially improve recognition of CLP speech while reducing performance disparities across severity groups. The proposed approach further enables low-latency, Internet-independent speech interaction on edge devices, supporting more accessible and inclusive voice-based human-computer interaction for individuals with CLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。