用人类精修数据和自适应强化学习，提升图像编辑指令的准确性。
EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesis
- 先自动生成再人工修正，构建10万条高质量指令数据。
- 引入错误类型与严重度，设计自适应奖励机制提升模型表现。
- 在多个评测中超越闭源模型，关键错误率下降超60%。
高质量的图像对与精准编辑指令对指导图像编辑至关重要，但大规模构建此类训练三元组成本高昂。现有方法依赖视觉语言模型自动合成指令，但发现强模型仍难以准确描述图像间的视觉变化，常出现方向不一致、视角模糊和细粒度属性缺失三种典型错误。在400张图像对的人类评估中，多个开源模型关键错误率超过47.75%，导致大量合成指令无法用于下游训练。为此，我们提出EditCaption，一种两阶段后训练流水线：首先通过GLM自动标注、EditScore过滤与人工精修构建10万条监督微调数据；其次收集1万条人工标注的偏好对，每条拒绝指令标注主要错误类型与严重度。基于此，我们提出硬度自适应错误感知的DPO（HAE-DPO），其目标函数结合人类标注的严重度、错误类型与参考模型难度，动态调整惩罚项。实验显示，235B模型经SFT+HAE-DPO后，在Eval-400、HQ-Edit、ByteMorph-Bench三项基准上分别达到4.720、4.672、4.651分，优于Gemini-3-Pro。人类评估确认关键错误率从47.75%降至17.50%，正确率由41.75%升至70.25%，超越Gemini-3-Pro的66.00%。
原文摘要 · Abstract (English)
High-quality source-target image pairs with precise editing instructions are essential for instruction-guided image editing, yet constructing such training triplets at scale remains costly. Recent pipelines often rely on vision-language models to synthesize editing instructions automatically, but we find that strong VLMs still struggle to describe visual transformations between image pairs. In particular, they exhibit three recurring failure modes: orientation inconsistency, viewpoint ambiguity, and missing fine-grained attributes. In a human evaluation on 400 image pairs, several open-source VLM baselines produce critical-error rates above 47\%, making many synthesized instructions unsuitable for downstream training. To address this, we propose EditCaption, a two-stage post-training pipeline for image editing instruction synthesis. First, we construct a 100K supervised fine-tuning dataset through GLM-based auto-captioning, EditScore filtering, and human refinement. Second, we collect 10K human-annotated preference pairs, where each rejected instruction is labeled with its primary error type and severity. Based on this dataset, we propose Hardness-Adaptive Error-Aware DPO (HAE-DPO), a task-adapted DPO objective that introduces an adaptive margin based on human-labeled severity, failure-mode type, and reference-model hardness. Experiments across three benchmarks demonstrate that our 235B model with SFT+HAE-DPO achieves state-of-the-art performance among open-source and closed models, scoring 4.720 on Eval-400, 4.672 on HQ-Edit, and 4.651 on ByteMorph-Bench -- surpassing Gemini-3-Pro on all three. Human evaluation confirms critical error rates drop from 47.75\% to 17.50\%, with correct rates improving from 41.75\% to 70.25\%, surpassing Gemini-3-Pro (66.00\%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。