Chunk-wise Attention Transducer让语音转写更快更准，适合实时部署。
Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text
- 将音频分块处理，块内用交叉注意力建模局部对齐。
- 训练内存减少46.2%，推理速度提升1.69倍，准确率最高提升6.3%相对WER。
- 特别适合语音翻译任务，突破传统模型单调对齐限制。
我们提出一种新型RNN-T扩展模型——块级注意力变换器（CHAT），在保持流式处理能力的同时，将音频按固定大小分块，并在每块内使用交叉注意力进行局部对齐建模。该方法显著降低RNN-T需处理的时间维度，带来明显效率提升：训练峰值内存最多减少46.2%，训练速度最快提升1.36倍，推理速度最快提升1.69倍。同时，在多种语言和任务上，CHAT均实现稳定精度提升：语音识别的相对词错率（WER）最高降低6.3%，语音翻译的BLEU值最高提升18.0%。该方法在语音翻译中表现尤为突出，有效缓解了RNN-T因严格单调对齐带来的性能瓶颈。结果表明，CHAT为在不牺牲实时性前提下部署更强的流式语音模型提供了实用方案。
原文摘要 · Abstract (English)
We propose Chunk-wise Attention Transducer (CHAT), a novel extension to RNN-T models that processes audio in fixed-size chunks while employing cross-attention within each chunk. This hybrid approach maintains RNN-T's streaming capability while introducing controlled flexibility for local alignment modeling. CHAT significantly reduces the temporal dimension that RNN-T must handle, yielding substantial efficiency improvements: up to 46.2% reduction in peak training memory, up to 1.36X faster training, and up to 1.69X faster inference. Alongside these efficiency gains, CHAT achieves consistent accuracy improvements over RNN-T across multiple languages and tasks -- up to 6.3% relative WER reduction for speech recognition and up to 18.0% BLEU improvement for speech translation. The method proves particularly effective for speech translation, where RNN-T's strict monotonic alignment hurts performance. Our results demonstrate that the CHAT model offers a practical solution for deploying more capable streaming speech models without sacrificing real-time constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。