arXiv:2505.02370cs.CVcs.AI2025-05ICCV被引 20

通过修正和对比指令提升图像编辑的监督质量,效果远超现有方法。

SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing

  • 用视觉语言模型修正指令与图像对齐,解决标注噪声问题。
  • 引入正负指令对比,用三元组损失增强训练信号,性能显著提升。
  • 无需预训练或复杂模块,模型更小、数据需求少,适合实际应用。

由于手动收集准确的编辑数据困难,现有数据集通常依赖自动化方法构建,导致编辑指令与图像对之间的不匹配引发噪声监督信号。尽管近期工作尝试通过生成更高质量图像、在识别任务上预训练或引入视觉语言模型(VLMs)来改进编辑模型,但未能解决这一根本问题。本文提出新方案:为给定图像对构造更有效的编辑指令,包括修正指令以更好匹配原始图像对,并引入对比编辑指令以进一步增强其有效性。我们发现,编辑模型在不同推理步骤中表现出与文本无关的特定生成属性。基于这些先验属性,我们定义了统一指导原则,用于引导VLM修正指令。然而,部分复杂编辑场景仅靠修正指令无法解决。为此,我们构建包含正负指令的对比监督信号,使用三元组损失引入模型训练,进一步提升监督效果。本方法无需VLM模块或预训练任务,提供更直接高效的方式获得优质监督信号,为基于指令的图像编辑提供了新颖、简洁且有效的新解法。多基准测试结果表明,该方法显著优于现有方法。相比此前最先进方法SmartEdit,我们在Real-Edit基准上提升9.19%,训练数据减少30倍,模型规模缩小13倍。

原文摘要 · Abstract (English)

Due to the challenges of manually collecting accurate editing data, existing datasets are typically constructed using various automated methods, leading to noisy supervision signals caused by the mismatch between editing instructions and original-edited image pairs. Recent efforts attempt to improve editing models through generating higher-quality edited images, pre-training on recognition tasks, or introducing vision-language models (VLMs) but fail to resolve this fundamental issue. In this paper, we offer a novel solution by constructing more effective editing instructions for given image pairs. This includes rectifying the editing instructions to better align with the original-edited image pairs and using contrastive editing instructions to further enhance their effectiveness. Specifically, we find that editing models exhibit specific generation attributes at different inference steps, independent of the text. Based on these prior attributes, we define a unified guide for VLMs to rectify editing instructions. However, there are some challenging editing scenarios that cannot be resolved solely with rectified instructions. To this end, we further construct contrastive supervision signals with positive and negative instructions and introduce them into the model training using triplet loss, thereby further facilitating supervision effectiveness. Our method does not require the VLM modules or pre-training tasks used in previous work, offering a more direct and efficient way to provide better supervision signals, and providing a novel, simple, and effective solution for instruction-based image editing. Results on multiple benchmarks demonstrate that our method significantly outperforms existing approaches. Compared with previous SOTA SmartEdit, we achieve 9.19% improvements on the Real-Edit benchmark with 30x less training data and 13x smaller model size.

图像编辑指令对齐对比学习监督优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。