arXiv:2605.13122cs.CV2026-05

利用图像编辑模型的早期语义定位能力,实现零样本图像分割。

Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation

论文配图:Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation
图 1 · 摘自论文原文
  • 通过分析编辑模型在去噪初期的内部表示,发现前景背景分离性强。
  • 仅用一次去噪步骤即可生成精准分割掩码,无需完整图像重建。
  • 无需训练,适配于零样本场景下快速部署的视觉定位任务。

基于指令的图像编辑(IIE)模型能根据自然语言指令修改特定图像区域,这隐含了对编辑位置的语言引导视觉语义定位能力。本文研究发现,此类模型在最早去噪时间步即出现显著的前景-背景可分性,远早于可见图像变化发生。基于此,提出一种无需训练的框架,利用预训练编辑模型的中间表示进行零样本指代图像分割(RIS)。方法将定位分解为:基于注意力的空间先验(确定关注区域)与基于特征的语义区分(确定分割内容)。借助特征空间的可分性,仅通过单次去噪步骤即可生成准确分割掩码,无需完成完整图像合成。在RefCOCO、RefCOCO+和RefCOCOg上的实验表明,该方法优于现有零样本基线。

原文摘要 · Abstract (English)

Instruction-based image editing (IIE) models have recently demonstrated strong capability in modifying specific image regions according to natural language instructions, which implicitly requires identifying where an edit should be applied. This indicates that such models inherently perform language-conditioned visual semantic grounding. In this work, we investigate whether this implicit grounding can be leveraged for zero-shot referring image segmentation (RIS), a task that requires pixel-level localization of objects described by natural language expressions. Through systematic analysis, we reveal that strong foreground-background separability emerges in the internal representations of these models at the earliest denoising timestep, well before any visible image transformation occurs. Building on this insight, we propose a training-free framework that repurposes pretrained image editing models for RIS by exploiting their intermediate representations. Our approach decomposes localization into two complementary components: attention-based spatial priors that estimate where to focus, and feature-based semantic discrimination that determines what to segment. By leveraging feature-space separability, the framework produces accurate segmentation masks using only a single denoising step, without requiring full image synthesis. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that our method achieves superior performance over existing zero-shot baselines.

图像分割零样本语义定位编辑模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。