用大模型提升多语言音素转字形的准确率
Advancing LLM-based phoneme-to-grapheme for multilingual speech recognition
- 引入简化版SKM方法，缓解音素识别不确定性对生成的影响
- 在10种语言上将平均词错误率从10.56%降至7.66%
- 适合做多语种语音识别系统研发的工程师参考
基于音素的自动语音识别将识别过程分解为声学到音素（S2P）和音素到字形（P2G）两步，实现跨语言声学共享，同时将语言特异的拼写规则保留在独立模块中。尽管大语言模型（LLMs）在P2G任务中表现潜力巨大，但多语言场景下仍面临语言感知生成与严重跨语言数据不平衡的挑战。本文在十语言的CV-Lang10基准上研究了基于LLM的多语言P2G，考察了应对S2P不确定性的鲁棒性策略，包括DANP和简化版SKM（S-SKM）。S-SKM是一种蒙特卡洛近似方法，避免在P2G训练中使用基于CTC的S2P概率加权。通过鲁棒训练和低资源数据过采样，平均词错误率（WER）从10.56%降低至7.66%。
原文摘要 · Abstract (English)
Phoneme-based ASR factorizes recognition into speech-to-phoneme (S2P) and phoneme-to-grapheme (P2G), enabling cross-lingual acoustic sharing while keeping language-specific orthography in a separate module. While large language models (LLMs) are promising for P2G, multilingual P2G remains challenging due to language-aware generation and severe cross-language data imbalance. We study multilingual LLM-based P2G on the ten-language CV-Lang10 benchmark. We examine robustness strategies that account for S2P uncertainty, including DANP and Simplified SKM (S-SKM). S-SKM is a Monte Carlo approximation that avoids CTC-based S2P probability weighting in P2G training. Robust training and low-resource oversampling reduce the average WER from 10.56% to 7.66%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。