用语音转换生成失语症语音数据，提升低资源语言的语音识别能力
Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages
- 用英语失语症数据微调语音转换模型，保留说话人特征和语调畸变
- 将健康非英语语音转为类失语症语音，用于增强多语言语音识别模型
- 在西语、意语、泰米尔语上显著优于现成模型和传统数据增强方法
失语症语音的自动语音识别因数据稀缺而困难，尤其在非英语语言中。为此，我们基于英语失语症语音数据集UASpeech微调语音转换模型，以编码说话人特征和语调畸变，再将其应用于非英语健康语音（FLEURS）生成非英语类失语症语音。利用生成数据微调多语言语音模型Massively Multilingual Speech（MMS），在西班牙语PC-GITA、意大利语EasyCall和泰米尔语SSNCE上的评估表明，结合说话人与语调转换的语音转换方法显著优于原版MMS及速度/节拍扰动等传统数据增强技术。客观与主观分析进一步验证了生成语音确实模拟了失语症特征。
原文摘要 · Abstract (English)
Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conversion model on English dysarthric speech (UASpeech) to encode both speaker characteristics and prosodic distortions, then apply it to convert healthy non-English speech (FLEURS) into non-English dysarthric-like speech. The generated data is then used to fine-tune a multilingual ASR model, Massively Multilingual Speech (MMS), for improved dysarthric speech recognition. Evaluation on PC-GITA (Spanish), EasyCall (Italian), and SSNCE (Tamil) demonstrates that VC with both speaker and prosody conversion significantly outperforms the off-the-shelf MMS performance and conventional augmentation techniques such as speed and tempo perturbation. Objective and subjective analyses of the generated data further confirm that the generated speech simulates dysarthric characteristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。