用对应关系引导修复，让图像恢复更忠于参考图。
CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models
- 通过自注意力掩码和约束更新输入张量，显式建模参考图与目标图的几何对应关系。
- 在多个基线方法上提升参考图像忠实度，显著改善修复结果一致性。
- 无需训练，可直接插入现有扩散模型，适合需要高保真修复的场景。
参考图引导的图像修复任务中，需利用额外参考图像将受损目标图像恢复至原始状态。尽管基于扩散模型（如Stable Diffusion）的方法已实现简洁建模，但现有方法缺乏对参考图与目标图之间相关性的显式约束，导致修复结果对参考图的忠实度不足。本文提出CorrFill，一种无需训练的模块，通过在修复过程中估计并利用几何对应关系进行引导，结合自注意力层中的注意力掩码及约束优化目标函数，动态更新输入张量以增强对应感知。实验表明，CorrFill能显著提升多种基线扩散模型（包括最先进方法）的参考图像忠实度，有效改善修复质量。
原文摘要 · Abstract (English)
In the task of reference-based image inpainting, an additional reference image is provided to restore a damaged target image to its original state. The advancement of diffusion models, particularly Stable Diffusion, allows for simple formulations in this task. However, existing diffusion-based methods often lack explicit constraints on the correlation between the reference and damaged images, resulting in lower faithfulness to the reference images in the inpainting results. In this work, we propose CorrFill, a training-free module designed to enhance the awareness of geometric correlations between the reference and target images. This enhancement is achieved by guiding the inpainting process with correspondence constraints estimated during inpainting, utilizing attention masking in self-attention layers and an objective function to update the input tensor according to the constraints. Experimental results demonstrate that CorrFill significantly enhances the performance of multiple baseline diffusion-based methods, including state-of-the-art approaches, by emphasizing faithfulness to the reference images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。