融合分层注意力的Mamba-U-Net模型，提升跨语料语音增强效果。
Exploring Resolution-Wise Shared Attention in Hybrid Mamba-U-Nets for Improved Cross-Corpus Speech Enhancement
- 在U-Net中结合Mamba与多头注意力，按分辨率共享注意力权重
- 小模型在两个外部测试集上性能超越所有基线，参数减半、计算量更低
- 适合追求高效高泛化能力的语音增强应用
近期研究表明，结合Mamba与注意力机制的模型在跨语料语音增强任务中表现出色。同时，将Mamba融入U-Net结构可实现当前最优的增强性能，且显著降低模型规模与计算复杂度。受此启发，本文提出RWSA-MambaUNet——一种新型高效的混合模型，将Mamba与多头注意力嵌入U-Net结构以提升跨语料泛化能力。分辨率级共享注意力（RWSA）指在对应时频分辨率层级间共享注意力机制。所提最优RWSA-MambaUNet模型在两个域外测试集上达到当前最优泛化性能。值得注意的是，最小模型在域外DNS 2020测试集上，于PESQ、SSNR、ESTOI三项指标均优于所有基线；在域外EARS-WHAM_v2测试集上，于SSNR、ESTOI、SI-SDR三项指标也全面领先，同时模型参数少于一半，浮点运算量仅为极小部分。
原文摘要 · Abstract (English)
Recent advances in speech enhancement have shown that models combining Mamba and attention mechanisms yield superior cross-corpus generalization performance. At the same time, integrating Mamba in a U-Net structure has yielded state-of-the-art enhancement performance, while reducing both model size and computational complexity. Inspired by these insights, we propose RWSA-MambaUNet, a novel and efficient hybrid model combining Mamba and multi-head attention in a U-Net structure for improved cross-corpus performance. Resolution-wise shared attention (RWSA) refers to layerwise attention-sharing across corresponding time- and frequency resolutions. Our best-performing RWSA-MambaUNet model achieves state-of-the-art generalization performance on two out-of-domain test sets. Notably, our smallest model surpasses all baselines on the out-of-domain DNS 2020 test set in terms of PESQ, SSNR, and ESTOI, and on the out-of-domain EARS-WHAM_v2 test set in terms of SSNR, ESTOI, and SI-SDR, while using less than half the model parameters and a fraction of the FLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。