HAINAN统一了语音识别的自回归与非自回归推理,兼顾速度与准确率。
HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR
- 融合自回归与非自回归推理,支持灵活切换
- 非自回归模式效率媲美CTC,自回归模式精度超TDT和RNN-T
- 半自回归推理可进一步提准,计算开销极低
我们提出 Hybrid-Autoregressive INference TrANsducers (HAINAN),一种基于 Token-and-Duration Transducer (TDT) 的新型语音识别架构。通过随机掩码预测网络输出进行训练,HAINAN 支持完整的自回归推理以及无需预测器的非自回归推理。此外,我们提出一种新的半自回归推理范式:先以非自回归方式生成初始假设,再对每个词元并行化地基于初始结果重预测。在多语言、多数据集上的实验表明,非自回归模式下 HAINAN 的效率与 CTC 相当,自回归模式下则与 TDT 持平;精度方面,自回归 HAINAN 超过 TDT 与 RNN-T,非自回归版本显著优于 CTC。半自回归推理进一步提升精度,且计算开销极小,部分场景甚至超越 TDT。该模型在准确率与速度间表现出优异的灵活性,适合实际语音识别应用。
原文摘要 · Abstract (English)
We present Hybrid-Autoregressive INference TrANsducers (HAINAN), a novel architecture for speech recognition that extends the Token-and-Duration Transducer (TDT) model. Trained with randomly masked predictor network outputs, HAINAN supports both autoregressive inference with all network components and non-autoregressive inference without the predictor. Additionally, we propose a novel semi-autoregressive inference paradigm that first generates an initial hypothesis using non-autoregressive inference, followed by refinement steps where each token prediction is regenerated using parallelized autoregression on the initial hypothesis. Experiments on multiple datasets across different languages demonstrate that HAINAN achieves efficiency parity with CTC in non-autoregressive mode and with TDT in autoregressive mode. In terms of accuracy, autoregressive HAINAN outperforms TDT and RNN-T, while non-autoregressive HAINAN significantly outperforms CTC. Semi-autoregressive inference further enhances the model's accuracy with minimal computational overhead, and even outperforms TDT results in some cases. These results highlight HAINAN's flexibility in balancing accuracy and speed, positioning it as a strong candidate for real-world speech recognition applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。