arXiv:2410.18737cs.CV2024-10CVPR被引 21

修正扩散模型生成中的引导偏差,让条件生成更稳定可靠。

Rectified Diffusion Guidance for Conditional Generation

  • 提出ReCFG方法,放宽引导系数约束,使生成过程符合扩散理论
  • 导出闭式解,预计算系数不影响采样速度
  • 无需重训练,适配主流扩散模型如SD3、EDM2

Classifier-Free Guidance(CFG)通过将条件与无条件得分函数以和为1的两个系数组合,成为扩散模型采样中实用的技术。然而理论上,使用CFG进行去噪无法表示为逆扩散过程,可能隐含使用风险。本文重新审视CFG理论,严格证明不当的系数配置(即广泛采用的和为1版本)会导致生成分布的期望偏移。为纠正此问题,我们提出ReCFG,放松引导系数约束,使得去噪过程严格符合扩散理论。进一步证明,在给定引导强度下,该方法具有闭式解,可通过遍历观测数据预计算系数,几乎不影响采样速度。在真实数据上的实证表明,该后置设计可兼容现有最先进的扩散模型,包括类别条件(如ImageNet上的EDM2)和文本条件(如CC12M上的SD3)模型,且无需重新训练。代码已公开于https://github.com/thuxmf/recfg。

原文摘要 · Abstract (English)

Classifier-Free Guidance (CFG), which combines the conditional and unconditional score functions with two coefficients summing to one, serves as a practical technique for diffusion model sampling. Theoretically, however, denoising with CFG \textit{cannot} be expressed as a reciprocal diffusion process, which may consequently leave some hidden risks during use. In this work, we revisit the theory behind CFG and rigorously confirm that the improper configuration of the combination coefficients (\textit{i.e.}, the widely used summing-to-one version) brings about expectation shift of the generative distribution. To rectify this issue, we propose ReCFG with a relaxation on the guidance coefficients such that denoising with \method strictly aligns with the diffusion theory. We further show that our approach enjoys a \textbf{\textit{closed-form}} solution given the guidance strength. That way, the rectified coefficients can be readily pre-computed via traversing the observed data, leaving the sampling speed barely affected. Empirical evidence on real-world data demonstrate the compatibility of our post-hoc design with existing state-of-the-art diffusion models, including both class-conditioned ones (\textit{e.g.}, EDM2 on ImageNet) and text-conditioned ones (\textit{e.g.}, SD3 on CC12M), without any retraining. Code is available at \href{https://github.com/thuxmf/recfg}{https://github.com/thuxmf/recfg}.

扩散模型条件生成引导优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。