通过窗口并行推理加速语音识别，速度提升2.4倍且不损失准确率。
WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detection
- 用窗口并行处理多帧，跳过空白帧快速定位有效输出
- 贪婪解码下速度最高提升2.4倍，词错误率与原模型一致
- 适用于实时语音识别系统，尤其适合低延迟场景
我们提出窗口化非空白检测推理（WIND），一种新颖的RNN-T加速策略，在不牺牲模型精度的前提下显著提升推理速度。传统方法逐帧处理，而WIND在窗口内并行处理多帧，使模型能快速定位非空白预测，实现显著加速。我们实现了贪婪解码、带标签循环的批量贪婪解码，并提出一种新型束搜索解码方法。在多个数据集上测试显示，贪婪模式下速度最快提升2.4倍，词错误率（WER）与基线相同；束搜索算法在保持高精度的同时大幅提速。代码将开源。
原文摘要 · Abstract (English)
We propose Windowed Inference for Non-blank Detection (WIND), a novel strategy that significantly accelerates RNN-T inference without compromising model accuracy. During model inference, instead of processing frames sequentially, WIND processes multiple frames simultaneously within a window in parallel, allowing the model to quickly locate non-blank predictions during decoding, resulting in significant speed-ups. We implement WIND for greedy decoding, batched greedy decoding with label-looping techniques, and also propose a novel beam-search decoding method. Experiments on multiple datasets with different conditions show that our method, when operating in greedy modes, speeds up as much as 2.4X compared to the baseline sequential approach while maintaining identical Word Error Rate (WER) performance. Our beam-search algorithm achieves slightly better accuracy than alternative methods, with significantly improved speed. We will open-source our WIND implementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。