用多维度奖励提升指令图像编辑数据质量,让模型更准更保真。
Multi-Reward as Condition for Instruction-based Image Editing
- 用大视觉语言模型生成三维度评分(指令遵循、细节保留、生成质量)
- 构建2万条高质量奖励数据集RewardEdit20K,支持精准训练
- 将评分与反馈作为条件输入,适配真实复杂编辑场景
高质量训练三元组(指令、原始图、编辑图)对指令式图像编辑至关重要。现有主流数据集(如InsPix2Pix)依赖文生图模型(如Stable Diffusion、DALL-E)生成,但这些模型未针对图像编辑优化,导致指令遵循不准、细节丢失、生成伪影等问题。本文提出以多视角奖励数据替代优化真实图像质量的方案:1)基于GPT-4o等顶级大视觉语言模型,设计三维度量化评估体系,从指令遵循、细节保留、生成质量三个角度对编辑结果打分(0~5),并收集具体失败点文本反馈,构建高质量奖励数据集RewardEdit20K;2)提出新型训练框架,将奖励分数与文本描述编码为嵌入向量,作为辅助条件输入编辑模型的潜在空间和U-Net中,实现对不完美三元组的学习;3)构建包含真实照片与多样指令的挑战性评测基准Real-Edit。实验表明,该多奖励条件模型在InsPix2Pix与SmartEdit两个主流编辑流程上均优于无奖励基线。代码已开源。
原文摘要 · Abstract (English)
High-quality training triplets (instruction, original image, edited image) are essential for instruction-based image editing. Predominant training datasets (e.g., InsPix2Pix) are created using text-to-image generative models (e.g., Stable Diffusion, DALL-E) which are not trained for image editing. Accordingly, these datasets suffer from inaccurate instruction following, poor detail preserving, and generation artifacts. In this paper, we propose to address the training data quality issue with multi-perspective reward data instead of refining the ground-truth image quality. 1) we first design a quantitative metric system based on best-in-class LVLM (Large Vision Language Model), i.e., GPT-4o in our case, to evaluate the generation quality from 3 perspectives, namely, instruction following, detail preserving, and generation quality. For each perspective, we collected quantitative score in $0\sim 5$ and text descriptive feedback on the specific failure points in ground-truth edited images, resulting in a high-quality editing reward dataset, i.e., RewardEdit20K. 2) We further proposed a novel training framework to seamlessly integrate the metric output, regarded as multi-reward, into editing models to learn from the imperfect training triplets. During training, the reward scores and text descriptions are encoded as embeddings and fed into both the latent space and the U-Net of the editing models as auxiliary conditions. 3) We also build a challenging evaluation benchmark with real-world images/photos and diverse editing instructions, named Real-Edit. Experiments indicate that our multi-reward conditioned model outperforms its no-reward counterpart on two popular editing pipelines, i.e., InsPix2Pix and SmartEdit. Code is released at https://github.com/bytedance/Multi-Reward-Editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。