让大模型更懂冷门词，用小数据提升语音识别准确率
OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabulary
- 冻结预训练大模型参数，动态注入上下文词表信息
- 在LibriSpeech上使罕见词错误率降低11.6点，总错误率降0.9点
- 适合需要快速适配新领域词汇的低资源语音应用
语音基础模型（SFMs）如开放式耳语风格语音模型（OWSM）在大规模数据上训练，实现高精度自动语音识别。然而，即使此类模型仍难以准确识别罕见和未见词汇。虽然上下文偏置（CB）是提升此类词汇识别的有前景方法，但多数CB方法需从头训练，因缺乏预训练知识导致性能低于SFMs。本文将现有CB方法与OWSM v3.1结合，同时冻结其预训练参数。通过利用SFMs中嵌入的知识，所提方法在小数据条件下仍能实现有效上下文偏置，保留了SFMs的优势。实验结果表明，在LibriSpeech 100测试集上，该方法使偏置词错误率（B-WER）降低11.6点，整体词错误率（WER）提升0.9点，实时因子降低7.5%。
原文摘要 · Abstract (English)
Speech foundation models (SFMs), such as Open Whisper-Style Speech Models (OWSM), are trained on massive datasets to achieve accurate automatic speech recognition. However, even SFMs struggle to accurately recognize rare and unseen words. While contextual biasing (CB) is a promising approach to improve recognition of such words, most CB methods are trained from scratch, resulting in lower performance than SFMs due to the lack of pre-trained knowledge. This paper integrates an existing CB method with OWSM v3.1 while freezing its pre-trained parameters. By leveraging the knowledge embedded in SFMs, the proposed method enables effective CB while preserving the advantages of SFMs, even with a small dataset. Experimental results show that the proposed method improves the biasing word error rate (B-WER) by 11.6 points, resulting in a 0.9 point improvement in the overall WER while reducing the real-time factor by 7.5% compared to the non-biasing baseline on the LibriSpeech 100 test-clean set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。