让扩散模型逐步推理,解决复杂任务准确率超92%
EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models
- 用迭代思维引导模块激活大模型深层推理能力
- 在多个基准上平均准确率达92.1%,领先基线8.3个百分点
- 适合需要分步推理的视觉语言生成任务
近期多模态大语言模型(MLLM)被广泛集成到扩散框架中,主要作为文本编码器处理空间推理等复杂任务。然而该范式存在两大缺陷:(i) MLLM文本编码器推理深度不足,单步编码无法触发链式思考过程,难以提供精准指导;(ii) 推理引导在解码过程中保持不变,阻碍扩散模型将复杂指令逐步分解为可执行的去噪步骤。为此,我们提出内生链式思考(EndoCoT)框架:首先通过迭代思维引导模块,不断优化潜在思维状态,激活大模型的推理潜力;其次,引入终端思维对齐模块,确保推理轨迹与真实答案一致。该框架使MLLM编码器输出精细推理引导,推动扩散模型逐步执行并最终以分步方式完成复杂任务。在多个基准(如Maze、TSP、VSP和Sudoku)上,平均准确率达到92.1%,优于最强基线8.3个百分点。代码与数据集已公开于https://internlm.github.io/EndoCoT/。
原文摘要 · Abstract (English)
Recently, Multimodal Large Language Models (MLLMs) have been widely integrated into diffusion frameworks primarily as text encoders to tackle complex tasks such as spatial reasoning. However, this paradigm suffers from two critical limitations: (i) MLLMs text encoder exhibits insufficient reasoning depth. Single-step encoding fails to activate the Chain-of-Thought process, which is essential for MLLMs to provide accurate guidance for complex tasks. (ii) The guidance remains invariant during the decoding process. Invariant guidance during decoding prevents DiT from progressively decomposing complex instructions into actionable denoising steps, even with correct MLLM encodings. To this end, we propose Endogenous Chain-of-Thought (EndoCoT), a novel framework that first activates MLLMs' reasoning potential by iteratively refining latent thought states through an iterative thought guidance module, and then bridges these states to the DiT's denoising process. Second, a terminal thought grounding module is applied to ensure the reasoning trajectory remains grounded in textual supervision by aligning the final state with ground-truth answers. With these two components, the MLLM text encoder delivers meticulously reasoned guidance, enabling the DiT to execute it progressively and ultimately solve complex tasks in a step-by-step manner. Extensive evaluations across diverse benchmarks (e.g., Maze, TSP, VSP, and Sudoku) achieve an average accuracy of 92.1%, outperforming the strongest baseline by 8.3 percentage points. The code and dataset are publicly available at https://internlm.github.io/EndoCoT/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。