用Hydra替换注意力机制，提升语音增强的保真度与效率
Improving DF-Conformer Using Hydra For High-Fidelity Generative Speech Enhancement on Discrete Codec Token
- 用双向结构状态空间模型替代近似注意力，更精准建模长序列
- 在离散码本语音上实现更高保真度，优于原DF-Conformer
- 保持线性复杂度，适合长语音生成任务，适合语音重建场景
稀疏注意力变体DF-Conformer通过正交随机特征（FAVOR+）加速自注意力，结合扩张卷积扩大感受野，在语音增强中表现优异。本文提出用双向选择性结构状态空间模型（Hydra）取代FAVOR+，以消除其近似误差并维持线性复杂度。在基于离散码本的生成式语音增强模型Genhancer上，新方法显著优于原DF-Conformer，在保持高效的同时提升了语音保真度。
原文摘要 · Abstract (English)
The Dilated FAVOR Conformer (DF-Conformer) is an efficient variant of the Conformer architecture designed for speech enhancement (SE). It employs fast attention through positive orthogonal random features (FAVOR+) to mitigate the quadratic complexity associated with self-attention, while utilizing dilated convolution to expand the receptive field. This combination results in impressive performance across various SE models. In this paper, we propose replacing FAVOR+ with bidirectional selective structured state-space sequence models to achieve two main objectives:(1) enhancing global sequential modeling by eliminating the approximations inherent in FAVOR+, and (2) maintaining linear complexity relative to the sequence length. Specifically, we utilize Hydra, a bidirectional extension of Mamba, framed within the structured matrix mixer framework. Experiments conducted using a generative SE model on discrete codec tokens, known as Genhancer, demonstrate that the proposed method surpasses the performance of the DF-Conformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。