统一视觉与文本的思维链推理，实现更连贯的多模态理解。
Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Vision
- 设计宏观与微观两级思维链，降低计算开销。
- 在 WISE、RISE 等基准上达到当前最优性能。
- 仅需 8 张 A100 80GB 显卡即可高效训练。
思维链（CoT）推理已被广泛用于提升大语言模型能力，通过将复杂任务分解为一系列简单子任务。然而，将 CoT 扩展至视觉-语言推理仍面临挑战，尤其在于如何建模视觉状态的演变过程。现有方法常因难以捕捉视觉状态转移或架构碎片化导致推理轨迹不连贯。为此，我们提出 Uni-CoT：一种统一的思维链框架，可在单一模型中实现连贯且具象化的多模态推理。核心思想是利用兼具图像理解与生成能力的模型，对视觉内容进行推理并建模动态视觉状态。为克服统一模型训练的高成本难题，Uni-CoT 引入两级推理范式：宏观级 CoT 用于高层任务规划，微观级 CoT 负责子任务执行，显著减少计算负担。此外，采用交织式图文监督与多任务目标联合训练策略，有效支持两级推理。实验表明，Uni-CoT 在推理驱动图像生成基准 WISE 及编辑基准 RISE、KRIS 上均取得当前最优表现，并具备强泛化能力。所有实验仅需 8 张 A100 80GB GPU 即可完成。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning has been widely adopted to enhance Large Language Models (LLMs) by decomposing complex tasks into simpler, sequential subtasks. However, extending CoT to vision-language reasoning tasks remains challenging, as it often requires interpreting transitions of visual states to support reasoning. Existing methods often struggle with this due to limited capacity of modeling visual state transitions or incoherent visual trajectories caused by fragmented architectures. To overcome these limitations, we propose Uni-CoT, a Unified Chain-of-Thought framework that enables coherent and grounded multimodal reasoning within a single unified model. The key idea is to leverage a model capable of both image understanding and generation to reason over visual content and model evolving visual states. However, empowering a unified model to achieve that is non-trivial, given the high computational cost and the burden of training. To address this, Uni-CoT introduces a novel two-level reasoning paradigm: A Macro-Level CoT for high-level task planning and A Micro-Level CoT for subtask execution. This design significantly reduces the computational overhead. Furthermore, we introduce a structured training paradigm that combines interleaved image-text supervision for macro-level CoT with multi-task objectives for micro-level CoT. Together, these innovations allow Uni-CoT to perform scalable and coherent multi-modal reasoning. Furthermore, thanks to our design, all experiments can be efficiently completed using only 8 A100 GPUs with 80GB VRAM each. Experimental results on reasoning-driven image generation benchmark (WISE) and editing benchmarks (RISE and KRIS) indicates that Uni-CoT demonstrates SOTA performance and strong generalization, establishing Uni-CoT as a promising solution for multi-modal reasoning. Project Page and Code: https://sais-fuxi.github.io/projects/uni-cot/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。