用语义信息增强语音识别，提升低延迟流式识别准确率
SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition
- 通过上下文模块融合历史帧嵌入的语义信息
- 小块流式场景下词错误率显著降低
- 适合需要低延迟实时语音识别的场景
许多自动语音识别（ASR）应用需要对音频数据进行流式处理。在流式模式下，系统需在输入未完成时就开始转录，即只能使用有限或无未来上下文。相比离线模式，这种未来上下文的缺失会降低流式ASR性能，尤其是在低延迟约束下。本文提出SENS-ASR，通过将语义信息注入神经转换器来提升流式ASR的转录质量。该语义信息由一个上下文模块从已有历史帧嵌入中提取，并通过知识蒸馏方式，从一个在训练集转录文本上微调过的句子嵌入语言模型中学习。标准数据集上的实验表明，SENS-ASR在小块流式场景下显著降低了词错误率。
原文摘要 · Abstract (English)
Many Automatic Speech Recognition (ASR) applications require streaming processing of the audio data. In streaming mode, ASR systems need to start transcribing the input stream before it is complete, i.e., the systems have to process a stream of inputs with a limited (or no) future context. Compared to offline mode, this reduction of the future context degrades the performance of Streaming-ASR systems, especially while working with low-latency constraint. In this work, we present SENS-ASR, an approach to enhance the transcription quality of Streaming-ASR by reinforcing the acoustic information with semantic information. This semantic information is extracted from the available past frame-embeddings by a context module. This module is trained using knowledge distillation from a sentence embedding Language Model fine-tuned on the training dataset transcriptions. Experiments on standard datasets show that SENS-ASR significantly improves the Word Error Rate on small-chunk streaming scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。