arXiv:2606.01985cs.CV2026-06

用强化学习让图像编辑能连续多轮操作,错误不累积,效果更稳定。

MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching

论文配图:MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching
图 1 · 摘自论文原文
  • 基于流匹配的强化学习框架,支持多轮编辑决策优化。
  • 在FLUX.1-Kontext-dev上提升6.85分,显著优于现有开源模型。
  • 适合需要反复修改图像的设计师或内容创作者使用。

基于指令的图像编辑虽已取得突破,但多数模型仅适用于单轮编辑,难以应对多轮交互式编辑——用户根据模型前一轮输出逐步调整图像。问题根源在于:一旦某轮失败,整个序列即失效;且存在误差传播,暴露偏差导致错误不断累积。为此,我们提出MT-EditFlow,一种基于流匹配的强化学习框架,通过多轮视角与多奖励机制,统一适配GRPO和NFT类强化学习方法。系统分析并优化奖励信号,包括回合级聚合策略、VLM推理模式以权衡奖励偏差与方差、优势融合层级以防止奖励劫持。研究发现,将聚合优势广播至整个编辑轨迹,有效连接局部规划与全局任务成功。大量实验表明,MT-EditFlow显著提升多种基础模型表现。尤其在FLUX.1-Kontext-dev上,第三轮整体性能提升6.85分,超越Qwen-Image-Edit等先进开源模型。通过保持高边际成功率并降低暴露偏差,为视觉内容创作中更可靠自然的人机协作奠定基础。

原文摘要 · Abstract (English)

Recent breakthroughs in instruction-based image editing have captured significant attention, as models are now capable of handling real-world editing demands with the practicality required by everyday users. However, editing models trained primarily for single-turn edits often break down in multi-turn editing--the natural interactive setting where a user iteratively refines an image based on the model's own previous outputs. This failure stems from the all-or-nothing requirement, where a single failed turn compromises the entire sequence, and error propagation, where exposure bias leads to compounding editing errors. To address these challenges, we introduce MT-EditFlow, a flow-matching reinforcement learning framework designed to optimize reward signals for sequential image editing. MT-EditFlow integrates a multi-turn perspective with a multi-reward formulation to provide a unified structure applicable to both GRPO and NFT-based reinforcement learning methods. We systematically analyze and optimize the reward signal by investigating effective scoring strategies for turn-level aggregation, VLM reasoning modes to trade off reward bias and variance, and advantage fusion levels to prevent reward hacking. Our findings reveal that broadcasting the aggregated advantage across the entire editing trajectory effectively bridges the gap between local planning and global multi-turn task success. Extensive experiments demonstrate that MT-EditFlow significantly improves performance across diverse base models. Notably, it boosts FLUX.1-Kontext-dev by 6.85 points in turn-3 overall performance, surpassing state-of-the-art open-source models such as Qwen-Image-Edit. By maintaining high marginal success rates and reducing exposure bias, MT-EditFlow provides a foundation for more reliable and natural human-AI collaboration in visual content creation.

图像编辑强化学习多轮交互流匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。