统一建模频谱与空间信息,实现任意麦克风阵列的语音分离
UniArray: Unified Spectral-Spatial Modeling for Array-Geometry-Agnostic Speech Separation
- 用虚拟麦克风估计和空间字典学习融合频域特征
- 在多种阵列布局下,分离性能超越现有方法
- 适合需要通用阵列适配的语音增强场景
阵列几何无关语音分离(AGA-SS)旨在开发不依赖麦克风阵列结构的有效分离方法。传统方法采用无排列操作(如求和或注意力机制)提取空间信息,但常导致计算开销大或通道间交互中空间信息利用不充分,影响性能。为此,我们提出UniArray,摒弃传统交错处理方式。该方法包含三个核心组件:虚拟麦克风估计(VME)模块、特征提取与融合模块、分层双路径分离器。VME确保在不同通道数阵列上保持鲁棒性;特征提取与融合模块通过频段级特征提取和空间字典学习(SDL)模块,融合频率-通道特征,使分离器聚焦于融合后特征;分层双路径分离器同时建模时间与频率轴上的特征依赖关系,兼顾效率。实验表明,UniArray在已见与未见阵列几何下,均在SI-SDRi、WB-PESQ、NB-PESQ和STOI指标上优于当前最优方法。
原文摘要 · Abstract (English)
Array-geometry-agnostic speech separation (AGA-SS) aims to develop an effective separation method regardless of the microphone array geometry. Conventional methods rely on permutation-free operations, such as summation or attention mechanisms, to capture spatial information. However, these approaches often incur high computational costs or disrupt the effective use of spatial information during intra- and inter-channel interactions, leading to suboptimal performance. To address these issues, we propose UniArray, a novel approach that abandons the conventional interleaving manner. UniArray consists of three key components: a virtual microphone estimation (VME) module, a feature extraction and fusion module, and a hierarchical dual-path separator. The VME ensures robust performance across arrays with varying channel numbers. The feature extraction and fusion module leverages a spectral feature extraction module and a spatial dictionary learning (SDL) module to extract and fuse frequency-bin-level features, allowing the separator to focus on using the fused features. The hierarchical dual-path separator models feature dependencies along the time and frequency axes while maintaining computational efficiency. Experimental results show that UniArray outperforms state-of-the-art methods in SI-SDRi, WB-PESQ, NB-PESQ, and STOI across both seen and unseen array geometries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。