让AI像人一样思考再作画，提升视觉生成的通用性。
ThinkGen: Generalized Thinking for Visual Generation
- 用思维链引导图文生成，分步拆解任务意图
- 在多个基准上达到顶尖性能，跨场景适应性强
- 适合需要精细控制生成过程的研究与应用
多模态大模型的思维链（CoT）推理已能有效解决复杂理解任务，但其在生成任务中的应用仍处于初级阶段，且受限于特定场景的机制，难以泛化。本文提出ThinkGen，首个基于思维驱动的视觉生成框架，显式利用多模态大模型的CoT推理能力，覆盖多种生成场景。ThinkGen采用解耦架构，由预训练多模态大模型和扩散变换器（DiT）组成：前者根据用户意图生成定制指令，后者依据指令生成高质量图像。我们进一步设计可分离的GRPO训练范式（SepGRPO），在两个模块间交替进行强化学习。该灵活结构支持跨多样数据集联合训练，实现广泛生成场景下的有效思维链推理。大量实验表明，ThinkGen在多个生成基准上均达到稳健的领先性能。代码已开源：https://github.com/jiaosiyuu/ThinkGen
原文摘要 · Abstract (English)
Recent progress in Multimodal Large Language Models (MLLMs) demonstrates that Chain-of-Thought (CoT) reasoning enables systematic solutions to complex understanding tasks. However, its extension to generation tasks remains nascent and limited by scenario-specific mechanisms that hinder generalization and adaptation. In this work, we present ThinkGen, the first think-driven visual generation framework that explicitly leverages MLLM's CoT reasoning in various generation scenarios. ThinkGen employs a decoupled architecture comprising a pretrained MLLM and a Diffusion Transformer (DiT), wherein the MLLM generates tailored instructions based on user intent, and DiT produces high-quality images guided by these instructions. We further propose a separable GRPO-based training paradigm (SepGRPO), alternating reinforcement learning between the MLLM and DiT modules. This flexible design enables joint training across diverse datasets, facilitating effective CoT reasoning for a wide range of generative scenarios. Extensive experiments demonstrate that ThinkGen achieves robust, state-of-the-art performance across multiple generation benchmarks. Code is available: https://github.com/jiaosiyuu/ThinkGen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。