用合成语音生成多种发音，让语音识别模型零样本识别生僻词。
Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciation
- 通过合成语音生成目标词的多种发音，构建发音变体字典。
- 在Whisper模型上实现零样本识别，测试集错误率降低43%-44%。
- 适合需要识别罕见词或专有名词的语音识别场景。
上下文自动语音识别（ASR）系统能够识别词汇外（OOV）词汇，如命名实体或生僻词，但受限于训练数据不足及发音模糊或不一致。本文提出一种基于合成语音的多发音上下文偏置方法，在预训练的Whisper模型上实现零样本上下文ASR。具体而言，利用文本到语音（TTS）系统合成包含目标稀有词的多样化语音样本，并通过Whisper模型提取多个预测发音变体。这些变体序列被组织成前缀树（trie），在解码时以浅融合方式对束搜索路径赋予权重。最终识别出的任意变体均映射回原始罕见词。在LibriSpeech数据集上的评估显示，该方法在test-clean和test-other上分别将有偏词错误率（B-WER）降低43%和44%，同时保持无偏词错误率（U-WER）基本不变。
原文摘要 · Abstract (English)
Contextual automatic speech recognition (ASR) systems allow for recognizing out-of-vocabulary (OOV) words, such as named entities or rare words. However, it remains challenging due to limited training data and ambiguous or inconsistent pronunciations. In this paper, we propose a synthesis-driven multi-pronunciation contextual biasing method that performs zero-shot contextual ASR on a pretrained Whisper model. Specifically, we leverage text-to-speech (TTS) systems to synthesize diverse speech samples containing each target rare word, and then use the pretrained Whisper model to extract multiple predicted pronunciation variants. These variant token sequences are compiled into a prefix-trie, which assigns rewards to beam hypotheses in a shallow-fusion manner during beam-search decoding. Subsequently, any recognized variant is mapped back to the original rare word in the final transcription. The evaluation results on the LibriSpeech dataset show that our method reduces biased-word error rate (B-WER) by 43% on test-clean and 44% on test-other while maintaining unbiased-WER (U-WER) essentially unchanged.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。