用双向循环注意力替代多头注意力，实现长语音识别的高精度与高速度。
Accurate, fast, cheap: Choose three. Replacing Multi-Head-Attention with Bidirectional Recurrent Attention for Long-Form ASR
- 用双向循环注意力替代多头注意力，降低计算复杂度。
- 在长语音任务上达到与多头注意力相当的准确率，吞吐量提升44%。
- 提出方向丢弃机制，可灵活调节精度与速度，支持交替方向解码。
长时语音识别是日益受关注的研究领域。基于多头注意力（MHA）的语音识别模型因序列长度呈二次复杂度，难以适应长语音任务。本文基于近期研究中提出的线性复杂度循环注意力（RA）层，发现双向RA层在短时和长时任务中均可达到与MHA相当的准确率。我们提出了一个强基线模型——有限上下文注意力（LCA），并证明RA层在保持相同准确率的同时更高效。我们还设计了适用于长语音的训练范式，使RA性能进一步提升，相比LCA在44%更高的吞吐量下实现更优准确率。此外，我们提出方向丢弃（Direction Dropout）正则化方法，可提升准确率、精细控制精度/吞吐权衡，并支持新的交替方向解码模式，带来更高吞吐量。
原文摘要 · Abstract (English)
Long-form speech recognition is an application area of increasing research focus. ASR models based on multi-head attention (MHA) are ill-suited to long-form ASR because of their quadratic complexity in sequence length. We build on recent work that has investigated linear complexity recurrent attention (RA) layers for ASR. We find that bidirectional RA layers can match the accuracy of MHA for both short- and long-form applications. We present a strong limited-context attention (LCA) baseline, and show that RA layers are just as accurate while being more efficient. We develop a long-form training paradigm which further improves RA performance, leading to better accuracy than LCA with 44% higher throughput. We also present Direction Dropout, a novel regularization method that improves accuracy, provides fine-grained control of the accuracy/throughput trade-off of bidirectional RA, and enables a new alternating directions decoding mode with even higher throughput.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。