arXiv:2608.20161cs.AI2026-08

让指令图像编辑的两阶段模型更懂错在哪,自动分配改进责任

DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing

论文配图:DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing
图 1 · 摘自论文原文
  • 用多计划多渲染推演,判断该优化规划还是渲染模块
  • 在复杂编辑任务上,准确率提升显著,最高超基线12.7%
  • 适合需要精细推理的图像编辑场景,如复杂语义修改

基于指令的图像编辑采用规划-渲染架构:视觉语言模型(VLM)将指令转为编辑计划,扩散模型执行该计划。仅使用最终图像奖励训练效率低下,因为错误无法区分应优化规划模块还是渲染模块,且规划过程中的问题难以定位。本文提出DARS,一种双层信用分配强化学习框架。通过多计划多渲染推演,估计计划间与计划内奖励变异,实现软模块路由;推演平均奖励用于自适应课程设计。在规划器内部,四字段结构化推理输出支持前缀门控奖励与词元级优势重加权,将结果反馈转化为局部监督。在五个基准测试上,DARS在相同骨干、数据、奖励模型和推演预算下优于联合强化学习基线,尤其在需深度推理的编辑任务中表现更优。

原文摘要 · Abstract (English)

Instruction-based image editing uses a planner-renderer pipeline: a vision-language model (VLM) first converts the instruction into an edit plan, and a diffusion model then executes that plan. Training such systems with only final-image rewards is inefficient because a poor edit does not reveal whether additional optimization should place more emphasis on the planner or the renderer, and even planner-dominant cases remain difficult to localize within a free-form reasoning trace. We present DARS, a reinforcement learning framework for dual-level credit assignment in this two-stage setting. Across modules, multi-plan multi-render rollouts estimate between-plan and within-plan reward variability for soft module routing, while rollout mean rewards provide hardness estimates for an adaptive curriculum. Within the planner, a four-field structured reasoning output enables a prefix-gated reward and token-level advantage reweighting, turning outcome-level feedback into localized supervision. Experiments on five benchmarks show that DARS outperforms a Joint~RL baseline with the same backbone, data, reward model, and rollout budget, with the largest gains on reasoning-intensive edits.

图像编辑强化学习结构化推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。