一个模型同时支持离线与低延迟语音识别，提升实时性能
Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization
- 用带右上下文的分块注意力和动态分块卷积实现统一建模
- 引入模式一致性正则化，使流式识别准确率在低延迟下提升1.2%
- 适合需要统一部署语音识别系统的产品团队
统一自动语音识别（ASR）系统可降低开发与维护成本，但训练单一模型同时在离线与低延迟流式场景中表现良好仍具挑战。本文提出一种基于变换器（RNNT）的统一ASR框架，支持单个模型内同时进行离线与流式解码，采用带右上下文的分块注意力与动态分块卷积。为缩小离线与流式性能差距，提出高效的Triton实现的模式一致性正则化（MCR-RNNT），促进不同训练模式间的预测一致性。实验表明，该方法在保持离线性能的同时，显著提升低延迟下的流式识别准确率，并可扩展至更大模型与数据集。所提统一ASR框架及英文模型检查点已开源。
原文摘要 · Abstract (English)
Unification of automatic speech recognition (ASR) systems reduces development and maintenance costs, but training a single model to perform well in both offline and low-latency streaming settings remains challenging. We present a Unified ASR framework for Transducer (RNNT) training that supports both offline and streaming decoding within a single model, using chunk-limited attention with right context and dynamic chunked convolutions. To further close the gap between offline and streaming performance, we introduce an efficient Triton implementation of mode-consistency regularization for RNNT (MCR-RNNT), which encourages agreement across training modes. Experiments show that the proposed approach improves streaming accuracy at low latency while preserving offline performance and scaling to larger model sizes and training datasets. The proposed Unified ASR framework and the English model checkpoint are open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。