arXiv:2506.14434cs.SDcs.AI2025-06ACL被引 1

一个模型同时支持流式与非流式语音识别,提升效率。

Unifying Streaming and Non-streaming Zipformer-based ASR

  • 用动态右文掩码训练统一的Zipformer模型,兼顾未来上下文。
  • 相比非流式模型,词错误率降低7.9%,延迟几乎无增加。
  • 可灵活调节延迟与精度平衡,适合实际部署场景。

为降低语音识别模型的开发、训练和部署成本,我们提出一种统一框架,仅用一个端到端模型同时支持流式与非流式自动语音识别(ASR)。通过在Zipformer模型训练中引入动态右文掩码,利用未来上下文信息。由于其多尺度特性,该方法在Zipformer上比其他Conformer模型更有效。我们在Librispeech及大型内部对话数据集上训练不同版本的模型,并在跨多个领域的真实服务器-客户端环境中评估。结果表明,该策略使词错误率相对降低7.9%,用户感知延迟轻微下降。增加右文帧数后,流式模型性能接近非流式模型。该方法还能根据客户需求灵活调控延迟与准确率的权衡。

原文摘要 · Abstract (English)

There has been increasing interest in unifying streaming and non-streaming automatic speech recognition (ASR) models to reduce development, training, and deployment costs. We present a unified framework that trains a single end-to-end ASR model for both streaming and non-streaming applications, leveraging future context information. We propose to use dynamic right-context through the chunked attention masking in the training of zipformer-based ASR models. We demonstrate that using right-context is more effective in zipformer models compared to other conformer models due to its multi-scale nature. We analyze the effect of varying the number of right-context frames on accuracy and latency of the streaming ASR models. We use Librispeech and large in-house conversational datasets to train different versions of streaming and non-streaming models and evaluate them in a production grade server-client setup across diverse testsets of different domains. The proposed strategy reduces word error by relative 7.9\% with a small degradation in user-perceived latency. By adding more right-context frames, we are able to achieve streaming performance close to that of non-streaming models. Our approach also allows flexible control of the latency-accuracy tradeoff according to customers requirements.

语音识别流式处理模型统一延迟优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。