用GPU加速的n元语法模型，让语音识别更准更快
NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decoding
- 重构数据结构实现GPU并行计算，支持多种语音模型
- 在域外场景下减少50%以上贪心解码误差，仅增7%开销
- 适合工业级语音识别系统，兼顾速度与准确率
统计n元语法语言模型广泛用于自动语音识别（ASR）中的上下文偏置任务。然而，现有实现因并行化差导致计算效率低下，限制了其在工业场景的应用。本文重新设计统计n元语法语言模型的数据结构，实现面向GPU优化的快速并行推理。提出的NGPU-LM方法支持所有主流ASR模型类型——包括转换器、注意力编码器-解码器模型和CTC——在贪心解码中实现可定制化上下文偏置，计算开销低于7%。该方法可在域外场景下消除超过50%的贪心解码与束搜索之间的精度差距，同时避免束搜索带来的显著延迟。NGPU-LM已开源。
原文摘要 · Abstract (English)
Statistical n-gram language models are widely used for context-biasing tasks in Automatic Speech Recognition (ASR). However, existing implementations lack computational efficiency due to poor parallelization, making context-biasing less appealing for industrial use. This work rethinks data structures for statistical n-gram language models to enable fast and parallel operations for GPU-optimized inference. Our approach, named NGPU-LM, introduces customizable greedy decoding for all major ASR model types - including transducers, attention encoder-decoder models, and CTC - with less than 7% computational overhead. The proposed approach can eliminate more than 50% of the accuracy gap between greedy and beam search for out-of-domain scenarios while avoiding significant slowdown caused by beam search. The implementation of the proposed NGPU-LM is open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。