arXiv:2506.13301cs.CV2025-06IJCAI被引 1

利用预训练扩散模型的潜在关联知识,实现高效语义一致的点控图像编辑。

AttentionDrag: Exploiting Latent Correlation Knowledge in Pre-trained Diffusion Models for Image Editing

  • 通过重用U-Net自注意力机制学习的潜在关联知识,自动识别需编辑区域。
  • 无需重训练或迭代优化,编辑速度显著提升且保持语义一致性。
  • 自适应生成掩码,支持精准、上下文感知的交互式编辑,适合创意设计场景。

传统点控图像编辑方法依赖迭代隐空间优化或几何变换,处理效率低或难以捕捉图像内部语义关系。这些方法常忽略预训练扩散模型中蕴含的强大编辑能力。本文提出一种新型单步点控图像编辑方法AttentionDrag,充分利用预训练扩散模型中的潜在知识与特征相关性进行编辑。该框架在不进行大量重优化或重训练的情况下,实现语义一致且高质量的图像操作。具体而言,我们复用DDIM反演过程中U-Net模块自注意力机制学习到的潜在关联知识,自动识别并调整相关图像区域,确保语义有效性与一致性。同时,AttentionDrag自适应生成掩码以引导编辑过程,实现精确且上下文感知的修改。实验表明,该方法性能超越多数现有先进方法,且速度显著更快,为点控图像编辑提供更高效、语义连贯的解决方案。

原文摘要 · Abstract (English)

Traditional point-based image editing methods rely on iterative latent optimization or geometric transformations, which are either inefficient in their processing or fail to capture the semantic relationships within the image. These methods often overlook the powerful yet underutilized image editing capabilities inherent in pre-trained diffusion models. In this work, we propose a novel one-step point-based image editing method, named AttentionDrag, which leverages the inherent latent knowledge and feature correlations within pre-trained diffusion models for image editing tasks. This framework enables semantic consistency and high-quality manipulation without the need for extensive re-optimization or retraining. Specifically, we reutilize the latent correlations knowledge learned by the self-attention mechanism in the U-Net module during the DDIM inversion process to automatically identify and adjust relevant image regions, ensuring semantic validity and consistency. Additionally, AttentionDrag adaptively generates masks to guide the editing process, enabling precise and context-aware modifications with friendly interaction. Our results demonstrate a performance that surpasses most state-of-the-art methods with significantly faster speeds, showing a more efficient and semantically coherent solution for point-based image editing tasks.

图像编辑扩散模型注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。