用变分推断让扩散模型更准地修复大块缺失图像
VIPaint: Image Inpainting with Pre-Trained Diffusion Models via Variational Inference
- 通过分层变分推断逼近真实条件分布
- 在大区域缺失时仍生成高质量多样修复结果
- 适用于去模糊、超分辨率等其他图像修复任务
扩散概率模型通过逐步去噪从高斯噪声生成新数据（如图像）。然而，将生成过程条件化于损坏或遮蔽图像颇具挑战。尽管已有多种方法利用扩散先验进行图像修复，但常无法从真实条件分布中采样，尤其在大范围遮蔽时表现不佳。许多基线方法也不适用于生成高质量图像且计算成本更低的潜在扩散模型。本文提出一种分层变分推断算法，优化了对真实扩散后验的非高斯马尔可夫近似。所提出的VIPaint方法在图像修复任务中优于现有方法，即使在最先进的文本条件潜在扩散模型下，也能对大区域遮蔽生成高质量、多样化的修复结果，并在去模糊和超分辨率等其他逆问题中同样有效。
原文摘要 · Abstract (English)
Diffusion probabilistic models learn to remove noise added during training, generating novel data (e.g., images) from Gaussian noise through sequential denoising. However, conditioning the generative process on corrupted or masked images is challenging. While various methods have been proposed for inpainting masked images with diffusion priors, they often fail to produce samples from the true conditional distribution, especially for large masked regions. Many baselines also cannot be applied to latent diffusion models which generate high-quality images with much lower computational cost. We propose a hierarchical variational inference algorithm that optimizes a non-Gaussian Markov approximation of the true diffusion posterior. Our VIPaint method outperforms existing approaches to inpainting, producing diverse high-quality imputations even for state-of-the-art text-conditioned latent diffusion models, and is also effective for other inverse problems like deblurring and superresolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。