arXiv:2511.09084eess.AS2025-11中稿 · regular paper in t…被引 1

提出块状注意力掩码机制,让语音识别更快更准。

Towards Effective and Efficient Non-autoregressive decoders for Conformer and LLM-based ASR using Block-based Attention Mask

  • 用块内并行、块间有序的方式实现非自回归解码
  • 在LS960上提速最高达2.31倍,错误率还更低
  • 适合追求高速高精度的语音识别系统部署

自动语音识别(ASR)系统通常依赖自回归(AR)Transformer解码器,因其序列特性限制了高效并行推理。为此,非自回归(NAR)方法旨在显著提升解码速度同时保持与AR基线相当的识别准确率。本文提出一种新型NAR块状注意力掩码解码器(AMD),有效提升解码效率并维持ASR准确率,且在Conformer和大语言模型(LLM)基ASR系统中均可灵活权衡性能与效率。AMD在连续输出块内实现并行推理,同时保持块间的单调左右预测。设计了一种单遍束搜索算法,动态融合连接时序分类(CTC)、AR解码器和AMD的概率。在标准语音LS960和老年人语音DBank数据集上进行了实验:a) Conformer编码器-解码器系统使用滤波器组输入特征;b) 融合WavLM特征;c) 进一步集成LLM解码器。在LS960任务中,所提三元解码器相比CTC + AR基线,在三种模型配置下分别获得最高1.44倍、1.55倍和2.31倍的解码加速比,且字错误率(WER)无统计显著增加。当实时因子(RTF)与基线相当时,三元解码器实现了0.19%、0.62%和0.13%的绝对WER降低(相对降幅分别为4.3%、16.3%和3.8%)。在DBank任务上也取得类似提升。

原文摘要 · Abstract (English)

Automatic speech recognition (ASR) systems often rely on autoregressive (AR) Transformer decoder architectures, which limit efficient inference parallelization due to their sequential nature. To this end, non-autoregressive (NAR) approaches aim primarily to achieve significant decoding speedup while the maintaining recognition accuracy that is comparable to AR baselines. This paper proposes a novel NAR block-based attention mask decoder (AMD) that effectively improves decoding efficiency while maintaining ASR accuracy, and also offering flexibility in balancing the performance-efficiency trade-off on both Conformer and large language model (LLM)-based ASR systems. The proposed AMD performs parallel inference within contiguous blocks of output labels while maintaining monotonic left-to-right prediction between blocks. A one-pass beam search algorithm is designed to dynamically fuse Connectionist Temporal Classification (CTC), AR decoder, and AMD probabilities. Experiments are conducted on normal speech LS960 and DBank elderly speech across: a) The Conformer encoder-decoder ASR system with filterbank input features; b) its integration with WavLM features; and c) further advancement by integrating an LLM-based decoder. On the LS960 task, the proposed AMD empowered tripartite decoder achieves decoding speedup ratios of up to 1.44x, 1.55x, and 2.31x under the three model configurations over the CTC + AR baselines, without statistically significant WER increases. When operating with real-time factors (RTFs) comparable to the baselines, the tripartite decoder produces statistically significant WER reductions of 0.19%, 0.62% and 0.13% absolute (4.3%, 16.3%, and 3.8% relative). Similar improvements are also obtained on the DBank task.

语音识别非自回归加速推理块注意力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。