用非自回归编辑字幕的方式实现快速语音识别，适合实时应用。
NLE: Non-autoregressive LLM-based ASR by Transcript Editing
- 将语音识别转为条件字幕编辑，支持完全并行生成。
- 在 Open ASR 上平均字错误率 5.67%，实时因子达 1630。
- 单句识别快 27 倍，适合低延迟场景。
虽然自回归（AR）的基于大语言模型的语音识别系统精度高，但其序列解码限制了并行性并带来高延迟。我们提出 NLE，一种非自回归（NAR）方法，将语音识别建模为条件字幕编辑，实现完全并行预测。NLE 从预训练语音编码器提取声学嵌入和初始假设，再通过双向语言模型编辑器进行修正，该编辑器采用潜在对齐目标进行训练。交错填充策略利用 Transformer 的恒等映射特性，使模型专注于纠错而非完整重建。在 Open ASR 排行榜上，NLE++ 实现 5.67% 平均字错误率（WER），实时因子（RTFx）为 1630。在单句场景下，NLE 相较于 AR 基线实现 27 倍加速，适用于实时应用。
原文摘要 · Abstract (English)
While autoregressive (AR) LLM-based ASR systems achieve strong accuracy, their sequential decoding limits parallelism and incurs high latency. We propose NLE, a non-autoregressive (NAR) approach that formulates speech recognition as conditional transcript editing, enabling fully parallel prediction. NLE extracts acoustic embeddings and an initial hypothesis from a pretrained speech encoder, then refines the hypothesis using a bidirectional LLM editor trained with a latent alignment objective. An interleaved padding strategy exploits the identity mapping bias of Transformers, allowing the model to focus on corrections rather than full reconstruction. On the Open ASR leaderboard, NLE++ achieves 5.67% average WER with an RTFx (inverse real-time factor) of 1630. In single-utterance scenarios, NLE achieves 27x speedup over the AR baseline, making it suitable for real-time applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。