系统梳理音频生成中扩散模型的设计原则与实现方法。
Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
- 以得分建模为统一框架，整合多种生成思路
- 提出可复现的代码库，支持多类音频应用
- 适用于音频生成、语音增强等任务研究者
扩散模型已成为生成高质量、多样化样本的强大深度生成技术，广泛应用于包括音频在内的多个领域。尽管已有综述提供概览，但对具体设计选择的深入讨论仍不足，且音频扩散模型缺乏针对不同应用场景的设计实施指南与对比依据。本文以得分建模视角为统一框架，涵盖流匹配等近期方法，系统分析扩散模型的训练与采样流程，以及通过不同条件机制实现的音频应用。为促进可复现研究与快速原型开发，我们开源了统一代码库（https://github.com/gzhu06/AudioDiffuser），支持多种音频任务。通过三个案例研究——音频生成、语音增强与文本到语音合成，在标准数据集上进行了基准评估，验证了该框架的有效性。
原文摘要 · Abstract (English)
Diffusion models have emerged as powerful deep generative techniques, producing high-quality and diverse samples in applications in various domains including audio. While existing reviews provide overviews, there remains limited in-depth discussion of these specific design choices. The audio diffusion model literature also lacks principled guidance for the implementation of these design choices and their comparisons for different applications. This survey provides a comprehensive review of diffusion model design with an emphasis on design principles for quality improvement and conditioning for audio applications. We adopt the score modeling perspective as a unifying framework that accommodates various interpretations, including recent approaches like flow matching. We systematically examine the training and sampling procedures of diffusion models, and audio applications through different conditioning mechanisms. To provide an integrated, unified codebase and to promote reproducible research and rapid prototyping, we introduce an open-source codebase (https://github.com/gzhu06/AudioDiffuser) that implements our reviewed framework for various audio applications. We demonstrate its capabilities through three case studies: audio generation, speech enhancement, and text-to-speech synthesis, with benchmark evaluations on standard datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。