用图像对学习编辑表示，让AI更懂真实修改意图。
EditCLIP: Representation Learning for Image Editing
- 通过输入图与修改图联合编码，学习统一编辑表示
- 在示例编辑和自动评估任务上均超越现有方法
- 适合需要精准理解图像变化的视觉编辑场景
我们提出EditCLIP，一种面向图像编辑的新型表征学习方法。该方法通过联合编码输入图像及其修改后版本，有效捕捉两者间的变换关系。为验证其有效性，我们将其应用于两类任务：基于示例的图像编辑与自动化编辑评估。在基于示例的编辑中，将InstructPix2Pix中的文本指令替换为从参考图像对计算出的EditCLIP嵌入；实验表明，该方法在性能、效率与通用性上均优于当前最优方法。在自动化评估中，通过比较给定图像对的EditCLIP嵌入与文本指令或另一参考图像对的嵌入相似度，衡量编辑质量与结构保持程度；实验显示，EditCLIP比现有基于CLIP的指标更贴近人类判断，提供可靠的编辑质量评估。
原文摘要 · Abstract (English)
We introduce EditCLIP, a novel representation-learning approach for image editing. Our method learns a unified representation of edits by jointly encoding an input image and its edited counterpart, effectively capturing their transformation. To evaluate its effectiveness, we employ EditCLIP to solve two tasks: exemplar-based image editing and automated edit evaluation. In exemplar-based image editing, we replace text-based instructions in InstructPix2Pix with EditCLIP embeddings computed from a reference exemplar image pair. Experiments demonstrate that our approach outperforms state-of-the-art methods while being more efficient and versatile. For automated evaluation, EditCLIP assesses image edits by measuring the similarity between the EditCLIP embedding of a given image pair and either a textual editing instruction or the EditCLIP embedding of another reference image pair. Experiments show that EditCLIP aligns more closely with human judgments than existing CLIP-based metrics, providing a reliable measure of edit quality and structural preservation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。