用扩散模型无监督分离语音，靠说话人嵌入保持身份一致
Unsupervised Single-Channel Speech Separation with Diffusion under Speaker-Embedding Guidance
- 用无监督扩散模型建模清晰语音，把分离当逆问题求解
- 引入说话人嵌入引导，让不同说话人更分开、同一人更连贯
- 适合研究无监督语音分离或扩散模型应用的开发者
语音分离是音频处理的基础任务，传统方法依赖配对混合语音的有监督训练，但通常使用合成数据，难以反映真实场景。本文重访源模型范式，仅在无混响语音上训练扩散生成模型，并将分离问题形式化为扩散逆问题。然而，无条件扩散模型缺乏说话人级条件控制，虽能捕捉局部声学结构，却导致分离语音在时间上说话人身份不一致。为此，我们提出说话人嵌入引导机制，在反向扩散过程中保持每个分离声道内的说话人一致性，同时推动不同说话人嵌入相互远离。此外，我们设计了一种面向分离任务的新求解器。大量实验验证，两种策略显著提升了无监督源模型语音分离性能。音频样本与代码见 https://runwushi.github.io/UnSepDiff_demo。
原文摘要 · Abstract (English)
Speech separation is a fundamental task in audio processing, typically addressed with fully supervised systems trained on paired mixtures. While effective, such systems typically rely on synthetic data pipelines, which may not reflect real-world conditions. Instead, we revisit the source-model paradigm, training a diffusion generative model solely on anechoic speech and formulating separation as a diffusion inverse problem. However, unconditional diffusion models lack speaker-level conditioning, they can capture local acoustic structure but produce temporally inconsistent speaker identities in separated sources. To address this limitation, we propose Speaker-Embedding guidance that, during the reverse diffusion process, maintains speaker coherence within each separated track while driving embeddings of different speakers further apart. In addition, we propose a new separation-oriented solver tailored for speech separation, and both strategies effectively enhance performance on the challenging task of unsupervised source-model-based speech separation, as confirmed by extensive experimental results. Audio samples and code are available at https://runwushi.github.io/UnSepDiff_demo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。