统一框架让语音识别同时支持实时与非实时，效果更好且不增加延迟。
Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition
- 用联合训练让同一模型无缝切换实时与非实时模式。
- 在不同延迟条件下，实时识别准确率显著提升。
- 适合需要兼顾实时性与高精度的语音应用开发。
尽管大语言模型（LLM）与自动语音识别（ASR）系统的深度融合已显著提升识别准确率，但在低延迟实时场景中的部署仍具挑战。本文提出Uni-ASR，一种基于LLM的统一框架，集成非实时与实时语音识别能力。通过联合训练机制，系统可在不修改架构的前提下无缝切换两种模式。此外，引入上下文感知训练范式与协同设计的回退解码策略，可在不增加额外延迟的情况下提升实时识别准确率。实验结果表明，Uni-ASR在非实时模式下表现具有竞争力，并在多种延迟约束下的实时场景中展现出强有效性。
原文摘要 · Abstract (English)
Although the deep integration of the Automatic Speech Recognition (ASR) system with Large Language Models (LLMs) has significantly improved accuracy, the deployment of such systems in low-latency streaming scenarios remains challenging. In this paper, we propose Uni-ASR, a unified framework based on LLMs that integrates both non-streaming and streaming speech recognition capabilities. We propose a joint training paradigm that enables the system to seamlessly transition between two recognition modes without any architectural modifications. Furthermore, we introduce a context-aware training paradigm and a co-designed fallback decoding strategy, which can enhance streaming recognition accuracy without introducing additional latency. The experimental results demonstrate that Uni-ASR not only achieves competitive performance within non-streaming mode, but also demonstrates strong effectiveness in streaming scenarios under diverse latency constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。