arXiv:2604.04746cs.CV2026-04被引 4

让AI像人一样分步作画,每一步都基于前一步的视觉进展。

Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning

  • 分四步迭代生成:文本规划→视觉草图→文本反思→视觉精修
  • 通过逐阶段监督确保图像空间与语义一致性,提升生成可控性
  • 适合追求可解释、可控图像生成的研究者与设计师

人类绘画是逐步进行的:先规划整体布局,再勾勒粗略草图,审视后细化细节,且每一步都依赖于不断演化的视觉状态。本论文提出过程驱动的图像生成范式,将合成过程分解为思想与行动交织的多步推理轨迹。模型不一次性生成图像,而是在多个迭代中推进,每个迭代包含四个阶段:文本规划、视觉草图、文本反思和视觉精修。文本推理显式指导视觉状态的演化,而生成的视觉中间状态又反过来约束并锚定下一步的文本推理。核心挑战在于中间状态的模糊性:如何评估部分完成的图像?我们通过密集的逐阶段监督解决,同时施加双重约束:对视觉中间状态,保持空间与语义一致性;对文本中间状态,保留先前视觉知识的同时,使模型能识别并修正违背提示的内容。该方法使生成过程显式、可解释且可直接监督。我们在多种文本到图像生成基准上进行了实验验证。

原文摘要 · Abstract (English)

Humans paint images incrementally: they plan a global layout, sketch a coarse draft, inspect, and refine details, and most importantly, each step is grounded in the evolving visual states. However, can unified multimodal models trained on text-image interleaved datasets also imagine the chain of intermediate states? In this paper, we introduce process-driven image generation, a multi-step paradigm that decomposes synthesis into an interleaved reasoning trajectory of thoughts and actions. Rather than generating images in a single step, our approach unfolds across multiple iterations, each consisting of 4 stages: textual planning, visual drafting, textual reflection, and visual refinement. The textual reasoning explicitly conditions how the visual state should evolve, while the generated visual intermediate in turn constrains and grounds the next round of textual reasoning. A core challenge of process-driven generation stems from the ambiguity of intermediate states: how can models evaluate each partially-complete image? We address this through dense, step-wise supervision that maintains two complementary constraints: for the visual intermediate states, we enforce the spatial and semantic consistency; for the textual intermediate states, we preserve the prior visual knowledge while enabling the model to identify and correct prompt-violating elements. This makes the generation process explicit, interpretable, and directly supervisable. To validate proposed method, we conduct experiments under various text-to-image generation benchmarks.

图像生成分步推理可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。