用图拉普拉斯优化注意力，让文本编辑更精准不扭曲
LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing
- 用自注意力构建图像块关系图，优化跨注意力空间一致性
- 在PIE-Bench上显著优于现有方法，减少编辑伪影
- 适合需要精准局部修改的图像编辑任务
文本引导的图像编辑旨在根据自然语言指令修改图像特定区域，同时保持整体结构和背景真实性。现有方法利用扩散模型生成的交叉注意力图提取掩码以定位目标区域进行修改。但由于交叉注意力机制侧重语义相关性，难以维持图像完整性，导致空间不一致，产生编辑伪影和形变。本文提出LOCATEdit，通过基于图的方法，利用自注意力导出的图像块关系优化交叉注意力图，确保注意力在图像区域间平滑连贯，使修改仅限于指定对象并保留周围结构。LOCATEdit在PIE-Bench上持续且显著优于现有基线，展现出顶尖性能与广泛适用性。代码见https://github.com/LOCATEdit/LOCATEdit/
原文摘要 · Abstract (English)
Text-guided image editing aims to modify specific regions of an image according to natural language instructions while maintaining the general structure and the background fidelity. Existing methods utilize masks derived from cross-attention maps generated from diffusion models to identify the target regions for modification. However, since cross-attention mechanisms focus on semantic relevance, they struggle to maintain the image integrity. As a result, these methods often lack spatial consistency, leading to editing artifacts and distortions. In this work, we address these limitations and introduce LOCATEdit, which enhances cross-attention maps through a graph-based approach utilizing self-attention-derived patch relationships to maintain smooth, coherent attention across image regions, ensuring that alterations are limited to the designated items while retaining the surrounding structure. LOCATEdit consistently and substantially outperforms existing baselines on PIE-Bench, demonstrating its state-of-the-art performance and effectiveness on various editing tasks. Code can be found on https://github.com/LOCATEdit/LOCATEdit/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。