用人类反馈训练的智能体框架,自动修复图像编辑中的细节问题。
EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement

- 构建多层级智能体,模拟人类看图纠错的感知-推理-行动-评估流程。
- 在15000张编辑图像上验证,显著提升缺陷定位与诊断准确率。
- 适合需要高保真度图像生成的研究者与工业应用开发者。
近期文本引导图像编辑(TIE)模型进展显著,但编辑结果仍常出现物体不自然、光照不匹配等细粒度问题。现有修复方法或依赖昂贵的迭代重生成,或使用空间定位弱的视觉语言模型,易引发语义漂移和不可靠局部修正。为此,我们构建了包含15,000张图像的细粒度人类反馈数据集EditFHF-15K,涵盖12种TIE模型在43个编辑任务上的结果,标注了60,000个缺陷区域与80,000个失败区域,并附有文本解释及45,000条平均意见得分(MOS),用于评估感知质量、指令遵循性与视觉一致性。基于此,我们提出EditRefiner,一种分层、可解释且以人为中心的代理框架,将后编辑修正重构为类人感知-推理-行动-评估循环:感知代理检测缺陷上下文显著图;推理代理据此进行符合人类认知的诊断推断;行动代理根据推理结果规划并执行局部重编辑;评估代理判断修复效果并决定是否继续优化。大量实验表明,EditRefiner在缺陷定位、诊断准确率和人类感知一致性上均优于现有方法,确立了自校正、感知可靠的图像编辑新范式。代码已公开于https://github.com/IntMeGroup/EditRefiner。
原文摘要 · Abstract (English)
Recent text-guided image editing (TIE) models have made remarkable progress, yet edited images still frequently suffer from fine-grained issues such as unnatural objects, lighting mismatch, and unexpected changes. Existing refinement approaches either rely on costly iterative regeneration or employ vision-language models (VLMs) with weak spatial grounding, often resulting in semantic drift and unreliable local corrections. To address these limitations, we first construct EditFHF-15K, a dataset of fine-grained human feedback for edited images, comprising (1) 15K images from 12 TIE models spanning 43 editing tasks, (2) 60K annotated artifact regions and 80K editing failure regions, each accompanied by textual reasoning, and (3) 45K mean opinion scores (MOSs) assessing perceptual quality, instruction following, and visual consistency. Based on EditFHF-15K, we propose EditRefiner, a hierarchical, interpretable, and human-aligned agentic framework that reformulates post-editing correction as a human-like perception-reasoning-action-evaluation loop. Specifically, we introduce: (1) a perception agent that detects contextual saliency maps of artifacts and editing failures, (2) a reasoning agent that interprets these perceptual cues to perform human-aligned diagnostic inference, (3) an action agent that uses the reasoning output to plan and execute localized re-editing, and (4) an evaluation agent that assesses the re-edited image and guides the action agent on whether further refinements are required. Extensive experiments demonstrate that EditRefiner consistently outperforms state-of-the-art methods in distortion localization, diagnose accuracy and human perception alignment, establishing a new paradigm for self-corrective and perceptually reliable image editing. The code is available at https://github.com/IntMeGroup/EditRefiner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。