用扩散模型零样本分离声音，不需训练即可根据语言指令分轨。
DGMO: Training-Free Audio Source Separation through Diffusion-Guided Mask Optimization
- 测试时优化频谱掩码，利用预训练扩散模型生成先验。
- 在AudioSet上达到91.3%的SAR，接近有监督方法性能。
- 适合需要快速部署、无标注数据的音频分离场景。
语言查询音频源分离（LASS）通过自然语言查询实现开放词汇的声音分离。现有方法依赖特定任务的训练，本文探索预训练扩散模型（原用于音频生成）是否能无需额外训练即完成分离。我们提出一种无训练框架，利用生成先验实现零样本LASS。分析简单适配后发现存在模态特异性挑战。为此，提出扩散引导掩码优化（DGMO），一种测试时优化框架，通过精细化调整频谱掩码实现精准、输入对齐的分离。该方法有效复用预训练扩散模型进行源分离，在AudioSet数据集上取得91.3%的分离准确率（SAR），性能媲美有监督方法。本工作将扩散模型应用拓展至分离领域，建立零样本音频分离新范式。代码已公开。
原文摘要 · Abstract (English)
Language-queried Audio Source Separation (LASS) enables open-vocabulary sound separation via natural language queries. While existing methods rely on task-specific training, we explore whether pretrained diffusion models, originally designed for audio generation, can inherently perform separation without further training. In this study, we introduce a training-free framework leveraging generative priors for zero-shot LASS. Analyzing naive adaptations, we identify key limitations arising from modality-specific challenges. To address these issues, we propose Diffusion-Guided Mask Optimization (DGMO), a test-time optimization framework that refines spectrogram masks for precise, input-aligned separation. Our approach effectively repurposes pretrained diffusion models for source separation, achieving competitive performance without task-specific supervision. This work expands the application of diffusion models beyond generation, establishing a new paradigm for zero-shot audio separation. The code is available at: https://wltschmrz.github.io/DGMO/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。