arXiv:2606.13679cs.CV2026-06

让任意图像生成模型支持图文交替生成,提升叙事与推理能力。

InterleaveThinker: Reinforcing Agentic Interleaved Generation

论文配图:InterleaveThinker: Reinforcing Agentic Interleaved Generation
图 1 · 摘自论文原文
  • 设计多智能体管道,用规划与评议员协同控制图文生成顺序。
  • 在多个基准上达到Nano Banana和GPT-5水平,4步推理任务提升显著。
  • 适合需要复杂视觉叙事或推理的开发者与研究者使用。

近期图像生成模型在单图生成与编辑中展现出出色的逼真度与指令遵循能力,但受限于架构,无法实现图文序列交替生成,而这一能力在视觉叙事、引导与具身操作中至关重要。即使最新的开源统一多模态模型(UMMs)也表现有限。本文提出InterleaveThinker,首个可为任意现有图像生成器赋予交替生成能力的多智能体流水线。通过规划智能体组织图文输入序列,指导生成器每一步的操作;再由评议员评估输出,识别偏离指令的样本并优化指令以重新生成。为此,我们构建了Interleave-Planner-SFT-80k与Interleave-Critic-SFT-112k进行格式冷启动,并开发Interleave-Critic-RL-13k,基于GRPO强化生成轨迹中的逐步指令修正能力。由于一次交替生成可能涉及超过25次生成调用,全局优化计算成本过高,因此提出准确率奖励与逐步奖励,使单步强化学习有效引导整个生成轨迹。实验表明,InterleaveThinker显著提升多种图像生成器性能,在交替生成基准上达到与Nano Banana和GPT-5相当水平;更意外的是,其在基于推理的基准上也带来显著提升,例如在4步FLUX.2-klein上,对WISE和RISE任务均有显著增益。

原文摘要 · Abstract (English)

Recent image generators have demonstrated impressive photorealism and instruction-following capabilities in single-image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text-image sequence), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open-source Unified Multimodal Models (UMMs) exhibit limited performance in this regard. In this paper, we introduce InterleaveThinker, the first multi-agent pipeline designed to endow any existing image generator with interleaved generation capabilities. Specifically, we employ a planner agent to organize the image-text input sequence, instructing the image generator on the required execution at each step. Subsequently, we introduce a critic agent to evaluate the generator's outputs, identify samples that deviate from the planned instructions, and refine the instructions for regeneration. To implement this pipeline, we construct the Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k to perform a format cold-start. Then we develop Interleave-Critic-RL-13k to reinforce the step-wise instruction correction capability within a generation trajectory using GRPO. Since a single interleaved generation trajectory may involve over 25 generator calls, optimizing the entire trajectory is computationally impractical. Therefore, we propose accuracy reward and step-wise reward, allowing single-step RL to effectively guide the entire generation trajectory. The results show that InterleaveThinker improves performance across various image generators. On interleaved generation benchmarks, it achieves performance comparable to Nano Banana and GPT-5. Surprisingly, it also significantly enhances the base model on reasoning-based benchmarks; for example, on 4-step FLUX.2-klein, we observe substantial gains on WISE and RISE.

图像生成多智能体推理增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。