用Mamba模拟注意力机制，高效提取目标声音
Cross-attention Inspired Selective State Space Models for Target Sound Extraction
- 借鉴Transformer交叉注意力思路，改造Mamba模型结构
- 在两个基准方法上验证，提取效果接近传统Transformer
- 适合追求计算效率的语音分离任务开发者
Transformer模型中的交叉注意力模块广泛应用于目标声音提取任务，通过给定线索融合特征以提取感兴趣信号。尽管效果良好，但其计算效率较低。近期状态空间模型（如Mamba）在多项任务中展现出与Transformer相当的性能，同时显著降低计算复杂度。然而，由于Mamba无法捕捉不同序列间的依赖关系，限制了其在目标声音提取中的应用。本文提出CrossMamba模型，利用Mamba的隐式注意力机制，计算线索与音频混合信号之间的依赖关系。Mamba的计算可分解为查询、键和值三部分，本方法使用线索生成查询，音频混合信号生成键和值，遵循Transformer交叉注意力的核心思想。在两种代表性目标声音提取方法上的实验结果验证了CrossMamba的有效性。
原文摘要 · Abstract (English)
The Transformer model, particularly its cross-attention module, is widely used for feature fusion in target sound extraction which extracts the signal of interest based on given clues. Despite its effectiveness, this approach suffers from low computational efficiency. Recent advancements in state space models, notably the latest work Mamba, have shown comparable performance to Transformer-based methods while significantly reducing computational complexity in various tasks. However, Mamba's applicability in target sound extraction is limited due to its inability to capture dependencies between different sequences as the cross-attention does. In this paper, we propose CrossMamba for target sound extraction, which leverages the hidden attention mechanism of Mamba to compute dependencies between the given clues and the audio mixture. The calculation of Mamba can be divided to the query, key and value. We utilize the clue to generate the query and the audio mixture to derive the key and value, adhering to the principle of the cross-attention mechanism in Transformers. Experimental results from two representative target sound extraction methods validate the efficacy of the proposed CrossMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。