发现DPS实际在做最大后验推断,提出改进方法提升图像生成质量。
Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior
- DPS本质是最大化后验而非估计条件得分
- 改进后生成图像质量更高、多样性显著增强
- 仅用100张图和8小时训练轻量得分估计器
近期扩散模型被用于无需额外训练的逆问题求解,其中扩散后验采样(DPS)是最流行的方法之一。以往分析认为DPS通过近似条件得分实现后验采样,但本文在512x512 ImageNet图像上验证发现:DPS的条件得分估计与训练良好的条件扩散模型得分差异显著,甚至劣于无条件得分;其条件得分均值明显偏离零,不符合有效得分特性;且生成样本质量高但多样性低。这表明DPS更接近最大后验(MAP)而非条件得分估计。为此,我们提出改进:1)通过多步梯度上升与投影显式最大化后验;2)使用仅需100张图像和8 GPU小时训练的轻量条件得分估计器。大量实验显示改进显著提升DPS性能。源码见https://github.com/tongdaxu/Rethinking-Diffusion-Posterior-Sampling-From-Conditional-Score-Estimator-to-Maximizing-a-Posterior。
原文摘要 · Abstract (English)
Recent advancements in diffusion models have been leveraged to address inverse problems without additional training, and Diffusion Posterior Sampling (DPS) (Chung et al., 2022a) is among the most popular approaches. Previous analyses suggest that DPS accomplishes posterior sampling by approximating the conditional score. While in this paper, we demonstrate that the conditional score approximation employed by DPS is not as effective as previously assumed, but rather aligns more closely with the principle of maximizing a posterior (MAP). This assertion is substantiated through an examination of DPS on 512x512 ImageNet images, revealing that: 1) DPS's conditional score estimation significantly diverges from the score of a well-trained conditional diffusion model and is even inferior to the unconditional score; 2) The mean of DPS's conditional score estimation deviates significantly from zero, rendering it an invalid score estimation; 3) DPS generates high-quality samples with significantly lower diversity. In light of the above findings, we posit that DPS more closely resembles MAP than a conditional score estimator, and accordingly propose the following enhancements to DPS: 1) we explicitly maximize the posterior through multi-step gradient ascent and projection; 2) we utilize a light-weighted conditional score estimator trained with only 100 images and 8 GPU hours. Extensive experimental results indicate that these proposed improvements significantly enhance DPS's performance. The source code for these improvements is provided in https://github.com/tongdaxu/Rethinking-Diffusion-Posterior-Sampling-From-Conditional-Score-Estimator-to-Maximizing-a-Posterior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。