用WavLM提取的离散语音特征提升语音识别上下文建模能力
Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR
- 用WavLM离散特征补充跨话语和内部上下文信息
- 在Gigaspeech上实现11.14%最低测试WER，相对降低3.54%
- 开源代码可复现，适合语音识别研究者参考
基于自监督学习（SSL）的离散语音表征具有高度紧凑和领域适应性强的优点。本文将WavLM模型提取的离散语音特征作为额外的跨话语声学上下文特征，引入到Zipformer-Transducer语音识别系统中。在1000小时的Gigaspeech语料上，全面验证了用离散标记特征替代Fbank特征来建模跨话语上下文（前向与后向段落）、当前话语内部上下文，或两者同时建模的有效性。使用离散标记构建跨话语上下文的最优Zipformer-Transducer系统，在开发集和测试集上相比仅使用话语内上下文的基线系统，词错误率（WER）绝对降低0.32%至0.41%，相对降低2.78%至3.54%。最终获得的最低公开记录WER为开发集11.15%、测试集11.14%。相关工作已开源，可在https://github.com/open-creator/icefall/tree/master/egs/gigaspeech/Context_ASR获取。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable. In this paper, SSL discrete speech features extracted from WavLM models are used as additional cross-utterance acoustic context features in Zipformer-Transducer ASR systems. The efficacy of replacing Fbank features with discrete token features for modelling either cross-utterance contexts (from preceding and future segments), or current utterance's internal contexts alone, or both at the same time, are demonstrated thoroughly on the Gigaspeech 1000-hr corpus. The best Zipformer-Transducer system using discrete tokens based cross-utterance context features outperforms the baseline using utterance internal context only with statistically significant word error rate (WER) reductions of 0.32% to 0.41% absolute (2.78% to 3.54% relative) on the dev and test data. The lowest published WER of 11.15% and 11.14% were obtained on the dev and test sets. Our work is open-source and publicly available at https://github.com/open-creator/icefall/tree/master/egs/gigaspeech/Context\_ASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。