让扩散模型的引导权重随条件和时间动态变化,提升生成质量与对齐度。
Learn to Guide Your Diffusion Model
- 引导权重由条件、去噪起点和终点时间共同决定,实现自适应调整。
- 在图像生成中显著降低FID,文本到图像任务中提升图像与提示的匹配度。
- 支持基于奖励函数的引导,适用于需要精准控制生成内容的场景。
无分类器引导(CFG)是提升条件扩散模型生成样本感知质量的常用技术,通过线性组合条件与无条件得分估计并引入引导权重ω。固定的大权重虽能显著改善视觉效果,但常导致分布对齐变差。为此,本文学习连续的引导权重ω_{c,(s,t)},使其依赖于条件c、去噪起始时间t及目标时间s。通过最小化真实条件分布的噪声样本与引导扩散过程样本之间的分布差异来实现。进一步扩展框架以支持基于奖励函数R(x₀,c)的引导,该函数定义在干净数据和条件上。在低维模拟和高维图像生成中均验证了有效性,图像生成任务中FID指标下降;文本到图像应用中,使用CLIP分数作为奖励函数时,引导权重显著提升了图像与提示的对齐程度。
原文摘要 · Abstract (English)
Classifier-free guidance (CFG) is a widely used technique for improving the perceptual quality of samples from conditional diffusion models. It operates by linearly combining conditional and unconditional score estimates using a guidance weight $ω$. While a large, static weight can markedly improve visual results, this often comes at the cost of poorer distributional alignment. In order to better approximate the target conditional distribution, we instead learn guidance weights $ω_{c,(s,t)}$, which are continuous functions of the conditioning $c$, the time $t$ from which we denoise, and the time $s$ towards which we denoise. We achieve this by minimizing the distributional mismatch between noised samples from the true conditional distribution and samples from the guided diffusion process. We extend our framework to reward guided sampling, enabling the model to target distributions tilted by a reward function $R(x_0,c)$, defined on clean data and a conditioning $c$. We demonstrate the effectiveness of our methodology on low-dimensional toy examples and high-dimensional image settings, where we observe improvements in Fréchet inception distance (FID) for image generation. In text-to-image applications, we observe that employing a reward function given by the CLIP score leads to guidance weights that improve image-prompt alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。