用轻量快速解码器加速语音识别，同时减少重复错误。
Hybrid Decoding: Rapid Pass and Selective Detailed Correction for Sequence Models
- 先用快解码器快速生成结果，再由大模型选择性修正。
- 在LibriSpeech和GigaSpeech上达到更优或相当的词错误率。
- 推理速度提升超一倍，仅需微调新增解码器。
近年来，基于Transformer的编码器-解码器模型在多语言语音识别中表现优异。然而，解码器的自回归特性及模型规模带来了显著的推理瓶颈。此外，尽管罕见，重复现象仍会降低识别准确率。为此，我们提出一种新型混合解码方法，既加速推理又缓解重复问题。该方法在预训练编码器基础上附加一个轻量、快速的解码器。推理时，快速解码器先生成输出，随后由Transformer解码器验证并选择性修正。实验表明，在LibriSpeech和GigaSpeech测试集上，仅微调新增解码器，即可实现与基线相当或更优的词错误率（WER），且推理速度超过两倍提升。
原文摘要 · Abstract (English)
Recently, Transformer-based encoder-decoder models have demonstrated strong performance in multilingual speech recognition. However, the decoder's autoregressive nature and large size introduce significant bottlenecks during inference. Additionally, although rare, repetition can occur and negatively affect recognition accuracy. To tackle these challenges, we propose a novel Hybrid Decoding approach that both accelerates inference and alleviates the issue of repetition. Our method extends the transformer encoder-decoder architecture by attaching a lightweight, fast decoder to the pretrained encoder. During inference, the fast decoder rapidly generates an output, which is then verified and, if necessary, selectively corrected by the Transformer decoder. This results in faster decoding and improved robustness against repetitive errors. Experiments on the LibriSpeech and GigaSpeech test sets indicate that, with fine-tuning limited to the added decoder, our method achieves word error rates comparable to or better than the baseline, while more than doubling the inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。