arXiv:2601.03467cs.CV2026-01被引 10

用强化学习提升图像编辑的逻辑推理能力,让模型先思考再生成。

ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing

  • 引入思维链机制,分规划与反思阶段探索多种语义假设。
  • 在多个数据集上显著优于基线,指令遵循准确率提升18%以上。
  • 适合需要精准语义理解的图像编辑任务,如复杂场景重构。

基于统一多模态生成模型的指令驱动图像编辑发展迅速,但其底层视觉推理能力有限,导致在以推理为核心的编辑任务中表现不佳。强化学习(RL)虽被用于提升编辑质量,但仍面临三大挑战:(1)推理探索受限于去噪随机性;(2)奖励融合存在偏差;(3)基于视觉语言模型(VLM)的指令奖励不稳定。本文提出ThinkRL-Edit,一种以推理为中心的强化学习框架,将视觉推理与图像合成解耦,并拓展推理探索范围至去噪之外。为此,我们在在线采样前引入基于思维链(CoT)的推理采样,包含规划与反思阶段,促使模型在生成前探索多个语义假设并验证其合理性。为避免加权聚合失败,提出跨多奖励维度的无偏链偏好分组策略。此外,用二元检查清单替代区间评分,获得更精确、低方差且可解释的奖励信号。实验表明,该方法在推理导向的图像编辑任务中显著优于现有工作,生成结果既忠实于指令,又具视觉连贯性与语义合理性。

原文摘要 · Abstract (English)

Instruction-driven image editing with unified multimodal generative models has advanced rapidly, yet their underlying visual reasoning remains limited, leading to suboptimal performance on reasoning-centric edits. Reinforcement learning (RL) has been investigated for improving the quality of image editing, but it faces three key challenges: (1) limited reasoning exploration confined to denoising stochasticity, (2) biased reward fusion, and (3) unstable VLM-based instruction rewards. In this work, we propose ThinkRL-Edit, a reasoning-centric RL framework that decouples visual reasoning from image synthesis and expands reasoning exploration beyond denoising. To the end, we introduce Chain-of-Thought (CoT)-based reasoning sampling with planning and reflection stages prior to generation in online sampling, compelling the model to explore multiple semantic hypotheses and validate their plausibility before committing to a visual outcome. To avoid the failures of weighted aggregation, we propose an unbiased chain preference grouping strategy across multiple reward dimensions. Moreover, we replace interval-based VLM scores with a binary checklist, yielding more precise, lower-variance, and interpretable rewards for complex reasoning. Experiments show our method significantly outperforms prior work on reasoning-centric image editing, producing instruction-faithful, visually coherent, and semantically grounded edits.

图像编辑强化学习思维链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。