arXiv:2605.27924cs.CV2026-05

用语义差异+指令引导自动标注图像编辑区域,低成本生成大规模训练数据。

SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization

论文配图:SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization
图 1 · 摘自论文原文
  • 通过视觉主干的语义差分+双向跨模态优化,精准定位用户意图的编辑区域。
  • 在5个基准上提升12.20% F1、11.16% IoU,生成超110万张高质量标注图。
  • 可将公开编辑数据转化为通用可复用的训练资源,适合图像编辑与检测研究者。

文本驱动的图像编辑发展迅速,但可靠定位编辑区域仍需大量像素级标注的图像篡改定位(IML)训练数据,而低成本大规模获取此类数据仍是难题。我们发现:公开的编辑数据集包含数百万对结构相同的原始/编辑图像,仅缺像素级掩码。自动恢复这些掩码极具挑战:像素差分易被扩散模型引起的全图扰动淹没,仅依赖指令的定位又会遗漏编辑器产生的非预期副作用。为此提出SIGMA(语义差异-指令引导掩码标注器),在视觉基础主干中进行语义特征差分,并通过双向跨模态精炼将指令生成的空间先验注入视觉流,增强编辑意图区域的差异信号。SIGMA分两阶段训练:阶段一监督修复掩码;阶段二通过VAE往返噪声校准、EMA自训练及编辑噪声解耦损失,缓解扩散域偏差。SIGMA在五个基准上优于现有自动掩码生成器(F1+12.20%,IoU+11.16%)。应用于公开编辑语料库后,生成约110万张IML训练数据,使六种不同检测器在五个数据集上平均提升18.34% F1,将原本未使用的编辑数据转化为模型无关的监督资源。代码将在论文接受后开源。

原文摘要 · Abstract (English)

Text-driven image editing has advanced rapidly, but reliably localizing these manipulations requires image manipulation localization (IML) models trained on large pixel-annotated datasets, and there is still no low-cost way to obtain such training data at scale. We observe that these data already exist in disguise: public editing datasets contain millions of structurally identical (original, edited) pairs to IML training samples, lacking only pixel-level masks. Recovering these masks automatically is non-trivial: pixel differencing is overwhelmed by diffusion-induced perturbations across all pixels, and instruction-only grounding localizes only what the prompt describes, missing unintended editor side-effects. We propose SIGMA (Semantic-difference Instruction-Grounding Mask Annotator), which performs semantic-feature differencing in a vision foundation backbone and injects an instruction-derived spatial prior into this visual stream via bidirectional cross-modal refinement, amplifying the difference signal at intended-edit regions when the editor faithfully realizes user intent. SIGMA is trained in two complementary stages: Stage I supervises on inpainting masks; Stage II closes the diffusion-domain shift via VAE-roundtrip noise calibration, EMA self-training, and an edit-noise disentanglement loss. SIGMA outperforms existing automatic mask generators on five benchmarks (+12.20% F1, +11.16% IoU). When applied to public editing corpora, it produces a ~1.1M IML training set that improves six diverse detectors by +18.34% F1 across five datasets, turning previously unused editing data into a model-agnostic supervisory resource for IML. We'll release the full codebase as soon as the paper is accepted.

图像编辑掩码生成扩散模型数据标注

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。