用扩散模型实现高效高精度语音识别，支持并行解码。
MDM-ASR: Bridging Accuracy and Efficiency in ASR with Diffusion-Based Non-Autoregressive Decoding
- 基于掩码扩散模型构建非自回归语音识别框架，实现并行生成。
- 在多个基准上超越现有非自回归模型，接近强自回归基线性能。
- 适合追求高速推理且对精度要求高的语音识别应用场景。
在序列到序列的Transformer语音识别中，自回归（AR）模型虽准确但解码慢，而非自回归（NAR）模型虽可并行解码却性能下降。本文提出一种基于掩码扩散模型的原理性NAR ASR框架，将预训练语音编码器与条件于声学特征和部分掩码文本的Transformer扩散解码器结合，实现并行词元预测。为缓解训练-推理不匹配问题，引入迭代自我修正训练，使模型暴露于自身中间预测结果。还设计了具有位置偏置的熵有界置信度采样器以进一步提升性能。在多个基准上的实验表明，该方法持续优于现有NAR模型，且在保持并行解码效率的同时达到与强AR基线相当的性能。
原文摘要 · Abstract (English)
In sequence-to-sequence Transformer ASR, autoregressive (AR) models achieve strong accuracy but suffer from slow decoding, while non-autoregressive (NAR) models enable parallel decoding at the cost of degraded performance. We propose a principled NAR ASR framework based on Masked Diffusion Models to reduce this gap. A pre-trained speech encoder is coupled with a Transformer diffusion decoder conditioned on acoustic features and partially masked transcripts for parallel token prediction. To mitigate the training-inference mismatch, we introduce Iterative Self-Correction Training that exposes the model to its own intermediate predictions. We also design a Position-Biased Entropy-Bounded Confidence-based sampler with positional bias to further boost results. Experiments across multiple benchmarks demonstrate consistent gains over prior NAR models and competitive performance with strong AR baselines, while retaining parallel decoding efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。