融合频谱与空间信息,提升复杂环境下的语音提取鲁棒性
Spectral or spatial? Leveraging both for speaker extraction in challenging data conditions
- 同时利用频谱和空间特征,动态调整权重以应对干扰
- 在方向估计误差达30°时仍保持有效语音提取
- 适合参考信息不准确的实时语音分离场景
本文提出一种鲁棒的多通道语音提取算法,旨在应对参考信息不准确的问题。现有方法通常仅依赖频谱或空间线索,而本方法融合两者以增强稳定性。针对噪声混合信号及两个可能不可靠的线索,设计专用网络动态平衡二者贡献,必要时忽略低效信息。通过模拟推理时误差(使用简单到达方向估计器和噪声频谱注册过程)评估系统性能。实验表明,即使存在显著参考误差,所提模型仍能成功提取目标说话人。
原文摘要 · Abstract (English)
This paper presents a robust multi-channel speaker extraction algorithm designed to handle inaccuracies in reference information. While existing approaches often rely solely on either spatial or spectral cues to identify the target speaker, our method integrates both sources of information to enhance robustness. A key aspect of our approach is its emphasis on stability, ensuring reliable performance even when one of the features is degraded or misleading. Given a noisy mixture and two potentially unreliable cues, a dedicated network is trained to dynamically balance their contributions-or disregard the less informative one when necessary. We evaluate the system under challenging conditions by simulating inference-time errors using a simple direction of arrival (DOA) estimator and a noisy spectral enrollment process. Experimental results demonstrate that the proposed model successfully extracts the desired speaker even in the presence of substantial reference inaccuracies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。