用分频与跨频机制改进语音增强，让模型更懂高频细节
Improving Speech Enhancement by Cross- and Sub-band Processing with State Space Model
- 分频处理：将全频带按信息相似性切分为4个子带，独立加权降低模型负担
- 跨频恢复：引入频谱重建模块，缓解高频低能量信息遗忘问题
- 轻量高效：在DNS 2021上超越多个SOTA方法，参数更少
最近，以Mamba为代表的状态空间模型（SSM）在长序列建模任务中表现出色，包括语音增强。然而，由于子带特征差异显著，对所有子带使用相同的SSM会限制其推理能力。此外，在处理时频表示的每个时间帧时，SSM可能遗忘低能量的高频信息，导致高频结构恢复困难。为此，我们提出跨频与子带Mamba（CSMamba）。为帮助SSM灵活处理不同子带特征，我们设计了频带分割模块，根据信息相似性将全频带划分为四个宽度不同的子带，并为每个子带分配独立权重，从而减轻SSM的推理负担。同时，为缓解高频低能量信息被遗忘的问题，我们引入频谱重建模块，从多角度增强跨频特征表示。在DNS Challenge 2021数据集上的实验表明，CSMamba在三项客观评价指标上优于多个最先进的语音增强方法，且参数更少。
原文摘要 · Abstract (English)
Recently, the state space model (SSM) represented by Mamba has shown remarkable performance in long-term sequence modeling tasks, including speech enhancement. However, due to substantial differences in sub-band features, applying the same SSM to all sub-bands limits its inference capability. Additionally, when processing each time frame of the time-frequency representation, the SSM may forget certain high-frequency information of low energy, making the restoration of structure in the high-frequency bands challenging. For this reason, we propose Cross- and Sub-band Mamba (CSMamba). To assist the SSM in handling different sub-band features flexibly, we propose a band split block that splits the full-band into four sub-bands with different widths based on their information similarity. We then allocate independent weights to each sub-band, thereby reducing the inference burden on the SSM. Furthermore, to mitigate the forgetting of low-energy information in the high-frequency bands by the SSM, we introduce a spectrum restoration block that enhances the representation of the cross-band features from multiple perspectives. Experimental results on the DNS Challenge 2021 dataset demonstrate that CSMamba outperforms several state-of-the-art (SOTA) speech enhancement methods in three objective evaluation metrics with fewer parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。