融合局部与全局注意力，提升语音分离效果
TF-MossFormer: Integrating Convolution Gated Local-Global Attentions for Enhanced Time-Frequency Domain Monaural Speech Separation
- 采用动态滑窗注意力，自适应调整局部感受野
- 在WSJ0-2Mix上达到24.4dB的SI-SDRi性能
- 适合追求高精度单声道语音分离的研究者
Transformer的全局注意力虽能捕捉长程依赖，却可能忽略语音分离中至关重要的细粒度局部连续性。本文提出TF-MossFormer，一种结合局部与全局注意力的时间-频率域变换器，用于单声道语音分离。其核心是内容感知的滑动窗口注意力机制，可动态调整感受野以增强局部交互，克服静态卷积的局限性。不同于基于时间分块的方法，TF-MossFormer利用二维谱图同时建模时序与频域结构。注意力层间引入卷积门控，优化特征选择与信息流动。在WSJ0-2Mix数据集上，参数量分别为590万、1690万和2540万时，SI-SDRi分别达到22.6、24.0和24.4 dB，优于已有方法。
原文摘要 · Abstract (English)
Transformers with global attention capture long-range dependencies but can miss the fine-grained local continuity crucial for speech separation. We propose TF-MossFormer, a time-frequency transformer that combines local and global attention to jointly model short- and long-range contexts for monaural speech separation. At its core is a content-aware sliding-window attention mechanism that dynamically adapts receptive fields for stronger local interactions, avoiding the rigidity of static convolutions. Unlike time-domain chunk-based methods, TF-MossFormer leverages the 2D spectrogram to model structure along both time and frequency axes. Convolutional gating between attention layers further improves feature selection and information flow. TF-MossFormer achieves SI-SDRi of 22.6, 24.0, and 24.4 dB on WSJ0-2Mix with 5.9M, 16.9M, and 25.4M parameters, respectively, outperforming prior approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。