一个模型同时支持流式与非流式语音识别,提升效率。
Unifying Streaming and Non-streaming Zipformer-based ASR
- 用动态右文掩码训练统一的Zipformer模型,兼顾未来上下文。
- 相比非流式模型,词错误率降低7.9%,延迟几乎无增加。
- 可灵活调节延迟与精度平衡,适合实际部署场景。
为降低语音识别模型的开发、训练和部署成本,我们提出一种统一框架,仅用一个端到端模型同时支持流式与非流式自动语音识别(ASR)。通过在Zipformer模型训练中引入动态右文掩码,利用未来上下文信息。由于其多尺度特性,该方法在Zipformer上比其他Conformer模型更有效。我们在Librispeech及大型内部对话数据集上训练不同版本的模型,并在跨多个领域的真实服务器-客户端环境中评估。结果表明,该策略使词错误率相对降低7.9%,用户感知延迟轻微下降。增加右文帧数后,流式模型性能接近非流式模型。该方法还能根据客户需求灵活调控延迟与准确率的权衡。
原文摘要 · Abstract (English)
There has been increasing interest in unifying streaming and non-streaming automatic speech recognition (ASR) models to reduce development, training, and deployment costs. We present a unified framework that trains a single end-to-end ASR model for both streaming and non-streaming applications, leveraging future context information. We propose to use dynamic right-context through the chunked attention masking in the training of zipformer-based ASR models. We demonstrate that using right-context is more effective in zipformer models compared to other conformer models due to its multi-scale nature. We analyze the effect of varying the number of right-context frames on accuracy and latency of the streaming ASR models. We use Librispeech and large in-house conversational datasets to train different versions of streaming and non-streaming models and evaluate them in a production grade server-client setup across diverse testsets of different domains. The proposed strategy reduces word error by relative 7.9\% with a small degradation in user-perceived latency. By adding more right-context frames, we are able to achieve streaming performance close to that of non-streaming models. Our approach also allows flexible control of the latency-accuracy tradeoff according to customers requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。