提出新方法实现高效零样本图像修复,推理速度更快且保持高画质。
Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance
- 用简单高斯后验替代复杂梯度计算,避免反向传播开销。
- 修复结果与微调模型相当,但推理耗时降低显著。
- 适合追求快速高质量图像修复的开发者或应用落地者。
扩散模型已成为图像编辑任务(如图像修复和局部修改)的强大先验,目标是生成与已知区域一致的逼真内容。特别是无需重训练的零样本方法,已展现出优异的重建效果。然而,现有先进方法依赖一系列代理似然函数,其梯度作为理想梯度的近似,在每一步逆过程都需通过去噪器计算向量-雅可比乘积,带来显著内存与运行时间开销。为此,我们提出一种新的似然代理,可生成简单的、易于采样的高斯后验转移,从而避开对去噪器网络的反向传播。大量实验表明,该方法在保持强观测一致性的同时,相比微调基线生成更连贯、高质量的修复结果,并大幅降低推理成本。代码已开源:https://github.com/YazidJanati/ding。
原文摘要 · Abstract (English)
Diffusion models have emerged as powerful priors for image editing tasks such as inpainting and local modification, where the objective is to generate realistic content that remains consistent with observed regions. In particular, zero-shot approaches that leverage a pretrained diffusion model, without any retraining, have been shown to achieve highly effective reconstructions. However, state-of-the-art zero-shot methods typically rely on a sequence of surrogate likelihood functions, whose scores are used as proxies for the ideal score. This procedure however requires vector-Jacobian products through the denoiser at every reverse step, introducing significant memory and runtime overhead. To address this issue, we propose a new likelihood surrogate that yields simple and efficient to sample Gaussian posterior transitions, sidestepping the backpropagation through the denoiser network. Our extensive experiments show that our method achieves strong observation consistency compared with fine-tuned baselines and produces coherent, high-quality reconstructions, all while significantly reducing inference cost. Code is available at https://github.com/YazidJanati/ding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。