无需训练即可实现结构与风格统一的图像修复方法
HarmonPaint: Harmonized Training-Free Diffusion Inpainting
- 利用自注意力机制中的掩码策略保持结构一致
- 通过扩散模型特性实现未遮挡区域到遮挡区的风格迁移
- 完全免训练，适用于多种场景和风格
现有图像修复方法通常需要大量重训练或微调才能无缝融合新内容，但在结构与风格一致性方面仍存在困难。针对这一问题，我们提出HarmonPaint，一种无需训练的修复框架，可直接嵌入扩散模型的注意力机制中，实现高质量、风格协调的图像修复。通过在自注意力中引入掩码策略，HarmonPaint在不进行模型重训练或微调的前提下，确保了修复区域的结构保真度；同时，利用扩散模型的内在特性，将未掩码区域的风格信息传递至掩码区域，实现风格上的自然融合。大量实验表明，HarmonPaint在多种场景与风格下均表现出色，验证了其通用性与高效性。
原文摘要 · Abstract (English)
Existing inpainting methods often require extensive retraining or fine-tuning to integrate new content seamlessly, yet they struggle to maintain coherence in both structure and style between inpainted regions and the surrounding background. Motivated by these limitations, we introduce HarmonPaint, a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models to achieve high-quality, harmonized image inpainting without any form of training. By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning. Additionally, we exploit intrinsic diffusion model properties to transfer style information from unmasked to masked regions, achieving a harmonious integration of styles. Extensive experiments demonstrate the effectiveness of HarmonPaint across diverse scenes and styles, validating its versatility and performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。