用预训练语音分类器做语音生成，省去额外模型，效果好还省资源。
Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generation
- 复用已有的语音分类器，加轻量模块实现条件生成。
- 在单模型中完成高质量语音合成，保持低内存与计算开销。
- 适合想简化生成系统、追求高效高质语音合成的研究者。
分类器引导是一种通过噪声条件分类器控制扩散生成过程的方法，使其向目标类别收敛。其缺点是需要独立训练分类器和扩散模型。为此，本文提出一种更紧凑的替代方案：将传统训练的语音分类器直接复用于扩散生成。从一个冻结的噪声条件分类器（在log-Mel空间）出发，添加一个轻量子网络，重用分类器中间表示，并仅在去噪得分匹配目标下训练该子网络。实验表明，预训练分类器可有效复用于条件生成，为判别建模与条件语音合成之间搭建了高效桥梁，实现高质量语音合成，同时显著降低内存占用和计算成本。
原文摘要 · Abstract (English)
Classifier guidance is a way to control diffusion generation by using a noise-conditioned classifier to steer the sampling process toward a target class. One drawback of classifier guidance is that it requires two separately trained models: a classifier and a diffusion model. We therefore study a more compact alternative in which a conventionally trained speech classifier is repurposed as the backbone for diffusion generation. Starting from a frozen noise-conditioned classifier in log-Mel space, we attach a lightweight subnetwork that reuses intermediate classifier representations and train only this subnetwork under a Denoising Score Matching objective. Our work shows that a pretrained classifier can be repurposed for conditional generation, providing an appealing bridge between discriminative modeling and conditional speech synthesis resulting in high speech quality within a single-backbone model, with reduced memory footprint and computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。