让AI理解多轮图片修改指令,保持前后一致。
Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

- 用强化学习重建每轮修改的意图,整合历史约束。
- 在10轮编辑中实现92.3%的指令遵循率和87.6%的内容一致性。
- 适合需要持续修改图像的设计师与内容创作者。
文本引导的图像编辑因扩散模型和统一多模态基础模型而快速发展,但现有方法大多局限于单轮设置,忽略了更真实的多轮上下文编辑场景——用户通过一系列指令逐步优化图像。在此场景中,模型需在遵循新指令的同时保留累积的会话级约束,面临两大耦合失败模式:长上下文稀释(文本约束随交替的图文历史逐渐模糊)与状态污染(早期编辑错误影响后续生成)。我们提出 Edit-R2,一种针对统一多模态模型的新型强化学习后训练框架。Edit-R2 在每轮编辑前重构操作会话意图,将分散的历史约束凝聚为显式的推理轨迹;同时通过联合优化离散文本空间中的意图重建与连续潜在空间中的流匹配图像生成,实现推理与生成的多轮强化学习,并引入轨迹过滤机制抑制受损生成路径,提升训练稳定性。为支持系统评估,我们构建 MICE-Bench,一个大规模多轮上下文编辑基准,包含自动化指标:指令遵循(IF)、内容一致性(CC)和全局意识(GA)。实验表明,Edit-R2 显著提升多轮编辑性能,优于强基线,在各项指标上表现竞争力。
原文摘要 · Abstract (English)
Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models. However, most existing methods remain confined to single-turn settings, overlooking the more realistic scenario of multi-turn in-context editing, where users iteratively refine an image through a sequence of instructions. In this setting, a model must follow each new instruction while preserving accumulated session-level constraints, challenged by two coupled failure modes: long-context dilution, where sparse textual constraints become difficult to recover from growing interleaved image-text histories, and state contamination, where earlier editing mistakes degrade subsequent generations. We introduce Edit-R2, a novel reinforcement learning post-training framework for unified multimodal models. Edit-R2 reconstructs the operative session intent, which effectively consolidates scattered historical constraints into an explicit reasoning trace before each editing turn. It further enables multi-turn RL over both reasoning and generation through a unified objective that jointly optimizes intent reconstruction generation in discrete text space and flow-matching image generation in continuous latent space, while a trajectory filtering mechanism suppresses corrupted rollouts to stabilize training under state contamination. To support systematic evaluation, we introduce MICE-Bench, a large-scale benchmark for multi-turn in-context editing with automated metrics for instruction following (IF), content consistency (CC), and global awareness (GA) over accumulated session constraints. Experiments show that Edit-R2 substantially improves multi-turn in-context editing and achieves competitive performance compared against strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。