arXiv:2505.21101cs.LGstat.ME2025-05被引 6

提出新采样方法,在保持多样性的同时提升生成质量。

Conditional Diffusion Models with Classifier-Free Gibbs-like Guidance

  • 基于瑞尼散度修正CFG,使其符合正确定义的扩散模型
  • 在图像与文本转音频任务中显著优于传统CFG
  • 适合需要高质量且多样样本的生成场景

Classifier-Free Guidance (CFG) 虽广泛用于提升条件扩散模型的生成质量与提示对齐度,但常导致样本多样性下降,形成质量与多样性的权衡。本文指出,CFG 实际上并不对应一个定义良好的扩散模型:在低噪声极限下,其生成分布并非由条件分布的幂次 w > 1 倾斜所得。关键缺失项为一项瑞尼散度项,作为排斥力以修正偏差,该修正项在低噪声时消失。基于此,我们提出类吉布斯采样策略,从无CFG的条件模型样本出发,迭代优化以逐步提升质量并维持多样性。在图像与文本转音频任务上验证,本方法在各项指标上均显著超越传统CFG。代码已开源。

原文摘要 · Abstract (English)

Classifier-Free Guidance (CFG) is a widely used technique for improving conditional diffusion models by linearly combining the outputs of conditional and unconditional denoisers. While CFG enhances visual quality and improves alignment with prompts, it often reduces sample diversity, leading to a challenging trade-off between quality and diversity. To address this issue, we make two key contributions. First, CFG generally does not correspond to a well-defined denoising diffusion model (DDM). In particular, contrary to common intuition, CFG does not yield samples from the target distribution associated with the limiting CFG score as the noise level approaches zero -- where the data distribution is tilted by a power $w \gt 1$ of the conditional distribution. We identify the missing component: a Rényi divergence term that acts as a repulsive force and is required to correct CFG and render it consistent with a proper DDM. Our analysis shows that this correction term vanishes in the low-noise limit. Second, motivated by this insight, we propose a Gibbs-like sampling procedure to draw samples from the desired tilted distribution. This method starts with an initial sample from the conditional diffusion model without CFG and iteratively refines it, preserving diversity while progressively enhancing sample quality. We evaluate our approach on both image and text-to-audio generation tasks, demonstrating substantial improvements over CFG across all considered metrics. The code is available at https://github.com/yazidjanati/cfgig

扩散模型生成质量多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。