用线性复杂度方法提升语音识别速度与效率
Linear Time Complexity Conformers with SummaryMixing for Streaming Speech Recognition
- 提出SummaryMixing改进的流式语音识别编码器
- 在流式与离线模式下均优于自注意力模型
- 训练解码更省算力，适合边缘设备部署
端到端语音识别（ASR）使用自注意力机制的编码器，无论是否流式处理，其时间复杂度随语音长度呈二次增长，导致训练和解码成本高，限制了在资源受限设备上的部署。SummaryMixing是一种有前景的线性时间复杂度替代方案，首次在非流式语音识别中保持或超越自注意力模型的精度。然而，原始的SummaryMixing不适用于流式场景。本文将其扩展至同时支持流式与离线模式的Conformer Transducer，结果表明该新编码器在两种模式下均优于自注意力模型，且训练与解码过程所需计算量和内存更少。
原文摘要 · Abstract (English)
Automatic speech recognition (ASR) with an encoder equipped with self-attention, whether streaming or non-streaming, takes quadratic time in the length of the speech utterance. This slows down training and decoding, increase their cost, and limit the deployment of the ASR in constrained devices. SummaryMixing is a promising linear-time complexity alternative to self-attention for non-streaming speech recognition that, for the first time, preserves or outperforms the accuracy of self-attention models. Unfortunately, the original definition of SummaryMixing is not suited to streaming speech recognition. Hence, this work extends SummaryMixing to a Conformer Transducer that works in both a streaming and an offline mode. It shows that this new linear-time complexity speech encoder outperforms self-attention in both scenarios while requiring less compute and memory during training and decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。