提出任务感知定位框架,精准控制图像编辑范围,避免无关区域被误改。
Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editing

- 基于源图和目标图的注意力线索,动态划分编辑与非编辑区域。
- 在EdiVal-Bench上显著提升非编辑区域一致性,保持强指令遵循能力。
- 无需训练,适配添加、删除、替换等多类编辑任务,适合高精度需求场景。
指令式图像编辑(IIE)旨在根据文本指令修改图像,同时保留无关内容。尽管扩散变换器取得进展,现有方法常出现过度编辑,对无关区域引入意外改动。我们发现,根本原因在于缺乏显式的编辑定位机制。不同操作(如添加、删除、替换)具有不同的空间模式,但现有模型通常采用通用定位策略。为此,我们提出一种无需训练的任务感知编辑定位框架,利用IIE模型中的源图与目标图流。首先获取注意力引导的编辑线索,再基于这些线索构建特征中心,将图像标记分为编辑与非编辑区域。鉴于最优定位本质上依赖任务,我们进一步设计统一掩码构造策略,针对不同任务选择性地使用源图或目标图流。通过系统分析验证了该方法的有效性。在EdiVal-Bench上的大量实验表明,本框架在保持强大指令遵循能力的同时,持续提升了非编辑区域的一致性,适用于Step1X-Edit与Qwen-Image-Edit等先进编辑模型。
原文摘要 · Abstract (English)
Instruction-based image editing (IIE) aims to modify images according to textual instructions while preserving irrelevant content. Despite recent advances in diffusion transformers, existing methods often suffer from over-editing, introducing unintended changes to regions unrelated to the desired edit. We identify that this limitation arises from the lack of an explicit mechanism for edit localization. In particular, different editing operations (e.g., addition, removal and replacement) induce distinct spatial patterns, yet current IIE models typically treat localization in a task-agnostic manner. To address this limitation, we propose a training-free, task-aware edit localization framework that exploits the intrinsic source and target image streams within IIE models. For each image stream, We first obtain attention-based edit cues, and then construct feature centroids based on these attentive cues to partition tokens into edit and non-edit regions. Based on the observation that optimal localization is inherently task-dependent, we further introduce a unified mask construction strategy that selectively leverages source and target image streams for different editing tasks. We provide a systematic analysis for our proposed insights and approaches. Extensive experiments on EdiVal-Bench demonstrate our framework consistently improves non-edit region consistency while maintaining strong instruction-following performance on top of powerful recent image editing backbones, including Step1X-Edit and Qwen-Image-Edit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。