用合成语音自循环训练，大幅提升印地语系语音识别准确率
The TTS-STT Flywheel: Synthetic Entity-Dense Audio Closes the Indic ASR Gap Where Commercial and Open-Source Systems Fail
- 构建TTS-STT自循环系统，生成2.2万条混合语言语音数据
- 在泰卢固语测试集上实体识别率提升至47.3%，超开源模型17倍
- 代码与数据开源，适合低资源语音识别研究者使用
小众领域的印地语系语音识别（如数字、货币、地址、品牌名及英印混写）在开源与商用系统中均表现不佳。在合成的实体密集型泰卢固语测试集上，开源最优模型vasista22/whisper-telugu-large-v2的实体命中率（EHR）仅为0.027，商用模型Deepgram Nova-3为0.16。本文提出自包含的TTS<->STT飞轮：利用开源印地语TTS系统生成约22,000条印地语-英语混写语音，成本低于50美元；在此基础上对vasista22进行LoRA微调，使泰卢固语测试集上EHR达0.473（相较开源模型提升17倍，商用模型3倍），且在FLEURS-Te上的读诵式语音识别错误率增加不超过6.6个百分点。跨语言表现：印地语β模型EHR达0.337（7倍于基线），泰米尔语β模型达0.543（22倍于基线，22倍于商用）。在深度语义覆盖较好的印地语上，飞轮性能略低于商用系统。三组β模型均未达预注册目标（泰卢固语0.75，印地语/泰米尔语0.65），结果如实报告。真人录音验证显示，飞轮在真实语音上表现稳定（β-Te EHR 0.516 vs 合成语音0.473）。消融实验表明，全部性能提升来自EDSA语料库（仅使用FLEURS-Te进行LoRA微调时EHR仅0.020）。此外发现：原生Whisper-large-v3存在泰卢固语特定字符坍塌（SFR 0.46-0.71），通过语言专属LoRA可纠正（SFR 0.81-0.97），但该方法不适用于印地语和泰米尔语（原生SFR ≥0.98）。代码、数据划分、预测结果、EDSA语料库与实体词典已开源。
原文摘要 · Abstract (English)
Niche-domain Indic ASR -- digit strings, currency amounts, addresses, brand names, English/Indic codemix -- is under-served by both open-source SOTA and commercial systems. On a synthesised entity-dense Telugu test set (held-out by synthesis system), vasista22/whisper-telugu-large-v2 (open SOTA) achieves Entity-Hit-Rate (EHR) 0.027 and Deepgram Nova-3 (commercial) 0.16. We close this gap with a self-contained TTS<->STT flywheel: an open-source Indic TTS pipeline synthesises ~22,000 entity-dense Indic-English code-mix utterances at <$50 marginal cost, and a LoRA fine-tune on top of vasista22 achieves EHR 0.473 on the held-out test (17x over open SOTA, 3x over commercial), with read-prose regression bounded to +6.6 pp WER on FLEURS-Te. Cross-language: beta-Hi 0.337 (7x vs vasista22) and beta-Ta 0.543 (22x vs vasista22, 22x vs Deepgram); on Hindi where Deepgram has substantial entity coverage, the flywheel underperforms commercial. All three beta models fall below pre-registered EHR targets (0.75 for Te, 0.65 for Hi/Ta); we report honestly. A native-human-recorded sanity check (n=20 Telugu) confirms transfer to real speech (beta-Te EHR 0.516 on native vs 0.473 on synth). An EDSA-isolation ablation (LoRA on FLEURS-Te alone) yields EHR 0.020 on the same held-out, attributing ~100% of the gain to the EDSA corpus. We additionally report a language-conditional finding: vanilla Whisper-large-v3 has Telugu-specific Script Collapse (SFR 0.46-0.71) that a per-language LoRA corrects (SFR 0.81-0.97), but the recipe is contraindicated on Hindi and Tamil where vanilla SFR >= 0.98. Code, holdouts, predictions, EDSA corpus, and entity dictionaries are released open-source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。