提升语音识别模型的解码速度与准确率，让高效推理成为可能。
Pushing the Limits of Beam Search Decoding for Transducer-based ASR models
- 采用批量运算与树状结构优化搜索过程
- 比贪婪解码快10%-20%，词错误率降低14%-30%
- 适合追求高精度与实时性的语音识别应用
基于变换器的端到端语音识别模型在识别准确率、流式处理和推理速度之间取得了良好平衡，但束搜索解码会因重复计算关键组件而显著降低速度，限制了实际应用。本文提出一种通用加速方法，实现了两种优化算法：ALSD++ 和 AES++。该方法结合批量操作、树状假设结构、新型空白分数设计以增强浅层融合，并利用 CUDA 图执行实现高效的 GPU 推理。实验表明，该方法将束搜索与贪婪模式的系统速度差距缩小至仅10%-20%，相比贪婪解码在词错误率（WER）上相对提升14%-30%，在低资源场景下浅层融合性能提升最高达11%。所有算法均已开源。
原文摘要 · Abstract (English)
Transducer models have emerged as a promising choice for end-to-end ASR systems, offering a balanced trade-off between recognition accuracy, streaming capabilities, and inference speed in greedy decoding. However, beam search significantly slows down Transducers due to repeated evaluations of key network components, limiting practical applications. This paper introduces a universal method to accelerate beam search for Transducers, enabling the implementation of two optimized algorithms: ALSD++ and AES++. The proposed method utilizes batch operations, a tree-based hypothesis structure, novel blank scoring for enhanced shallow fusion, and CUDA graph execution for efficient GPU inference. This narrows the speed gap between beam and greedy modes to only 10-20% for the whole system, achieves 14-30% relative improvement in WER compared to greedy decoding, and improves shallow fusion for low-resource up to 11% compared to existing implementations. All the algorithms are open sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。