arXiv:2411.10232cs.CVcs.AI2024-11被引 4

无需训练即可精准改图中物体颜色,解决文本编辑时颜色失效问题。

ColorEdit: Training-free Image-Guided Color editing with diffusion model

  • 通过分析扩散模型跨注意力层的特征,发现颜色调整可在去噪早期实现。
  • 利用值矩阵对齐技术,实现不需微调的图像引导颜色修改。
  • 提出首个颜色编辑基准数据集,适用于真实与合成图像的评估。

文本到图像(T2I)扩散模型凭借强大的生成能力被广泛用于图像编辑任务,但因物体与新颜色属性间的注意力泄漏和冲突,文本引导的编辑方法常无法改变物体颜色,导致结果与提示语不一致。本文深入分析了文本引导图像生成过程及不同跨注意力块所学习的语义信息,发现物体的视觉表示在去噪过程早期的上行模块中确定,而颜色调整可通过跨注意力层的值矩阵对齐实现。基于此,我们提出一种无需额外微调或训练的简单、稳定且有效的图像引导颜色修改方法。最后,我们构建了首个用于评估颜色变化方法性能的基准数据集COLORBENCH。大量实验验证了该方法在对象级颜色编辑中的有效性,在合成与真实图像上均优于主流文本引导编辑方法。

原文摘要 · Abstract (English)

Text-to-image (T2I) diffusion models, with their impressive generative capabilities, have been adopted for image editing tasks, demonstrating remarkable efficacy. However, due to attention leakage and collision between the cross-attention map of the object and the new color attribute from the text prompt, text-guided image editing methods may fail to change the color of an object, resulting in a misalignment between the resulting image and the text prompt. In this paper, we conduct an in-depth analysis on the process of text-guided image synthesizing and what semantic information different cross-attention blocks have learned. We observe that the visual representation of an object is determined in the up-block of the diffusion model in the early stage of the denoising process, and color adjustment can be achieved through value matrices alignment in the cross-attention layer. Based on our findings, we propose a straightforward, yet stable, and effective image-guided method to modify the color of an object without requiring any additional fine-tuning or training. Lastly, we present a benchmark dataset called COLORBENCH, the first benchmark to evaluate the performance of color change methods. Extensive experiments validate the effectiveness of our method in object-level color editing and surpass the performance of popular text-guided image editing approaches in both synthesized and real images.

图像编辑扩散模型颜色编辑零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。