用滑动窗口注意力实现低延迟语音识别,精度媲美大模型。
Moonshine v2: Ergodic Streaming Encoder ASR for Latency-Critical Speech Applications
- 采用滑动窗口自注意力机制,限制上下文范围以降低延迟。
- 在标准数据集上达到顶尖词错误率,精度与6倍大模型相当。
- 适合资源受限的边缘设备,适用于实时语音交互场景。
面向低延迟语音应用(如实时转录、语音指令、实时翻译),需兼顾快速首词输出(TTFT)与高准确率,尤其在资源受限的边缘设备上。全注意力Transformer编码器因可全局建模,仍为语音识别(ASR)的高精度基准,但其序列长度呈二次复杂度,导致必须完整编码语句才能生成首个输出,使首词延迟随语句增长线性上升。为更好满足边缘端流式ASR需求,本文提出Moonshine v2,一种基于滑动窗口自注意力的遍历式流式编码器模型,在保持强局部上下文能力的同时实现有界低延迟推理。该模型在标准基准上达到业界最优词错误率,性能与6倍大小的模型相当,且运行速度显著更快。结果表明,精心设计的局部注意力可仅以极小规模和延迟代价,达到全注意力的精度水平,为边缘设备上的交互式语音界面开辟新可能。
原文摘要 · Abstract (English)
Latency-critical speech applications (e.g., live transcription, voice commands, and real-time translation) demand low time-to-first-token (TTFT) and high transcription accuracy, particularly on resource-constrained edge devices. Full-attention Transformer encoders remain a strong accuracy baseline for automatic speech recognition (ASR) because every frame can directly attend to every other frame, which resolves otherwise locally ambiguous acoustics using distant lexical context. However, this global dependency incurs quadratic complexity in sequence length, inducing an inherent "encode-the-whole-utterance" latency profile. For streaming use cases, this causes TTFT to grow linearly with utterance length as the encoder must process the entire prefix before any decoder token can be emitted. To better meet the needs of on-device, streaming ASR use cases we introduce Moonshine v2, an ergodic streaming-encoder ASR model that employs sliding-window self-attention to achieve bounded, low-latency inference while preserving strong local context. Our models achieve state of the art word error rates across standard benchmarks, attaining accuracy on-par with models 6x their size while running significantly faster. These results demonstrate that carefully designed local attention is competitive with the accuracy of full attention at a fraction of the size and latency cost, opening new possibilities for interactive speech interfaces on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。