arXiv:2602.01608cs.AI2026-02

让自回归与扩散模型协同推理,提升生成可靠性与可控性

Reasoning with Autoregressive-Diffusion Collaborative Thoughts

  • 自回归模型规划约束,扩散模型生成视觉中间态,闭环反馈迭代优化
  • 在空间推理任务中显著降低错误传播,提升结构与物理合理性
  • 通用框架适用于问答与图像生成,适配多模态复杂任务场景

自回归模型擅长序列规划与约束组合,但在需要显式空间或物理依据的任务上表现不佳;扩散模型能捕捉丰富的空间结构,却缺乏分步逻辑控制以满足多阶段复杂约束或可靠纠错。本文提出协同思维(Collaborative Thoughts)框架,通过闭环交互使两类模型协同推理与生成。自回归模型执行结构化规划与约束管理,扩散模型将这些约束转化为中间视觉思维,基于视觉的评判模块评估其是否满足预期的结构与物理要求。该反馈用于迭代优化后续规划与生成步骤,缓解跨模态误差传播。重要的是,该协同循环在自回归问答与扩散视觉生成任务中均适用。通过代表性示例,展示了该方法在空间推理可靠性与生成可控性上的提升。

原文摘要 · Abstract (English)

Autoregressive and diffusion models represent two complementary generative paradigms. Autoregressive models excel at sequential planning and constraint composition, yet struggle with tasks that require explicit spatial or physical grounding. Diffusion models, in contrast, capture rich spatial structure through high-dimensional generation, but lack the stepwise logical control needed to satisfy complex, multi-stage constraints or to reliably identify and correct errors. We introduce Collaborative Thoughts, a unified collaborative framework that enables autoregressive and diffusion models to reason and generate jointly through a closed-loop interaction. In Collaborative Thoughts, autoregressive models perform structured planning and constraint management, diffusion models instantiate these constraints as intermediate visual thoughts, and a vision-based critic module evaluates whether the visual thoughts satisfy the intended structural and physical requirements. This feedback is then used to iteratively refine subsequent planning and generation steps, mitigating error propagation across modalities. Importantly, Collaborative Thoughts uses the same collaborative loop regardless of whether the task is autoregressive question answering or diffusion-based visual generation. Through representative examples, we illustrate how Collaborative Thoughts can improve the reliability of spatial reasoning and the controllability of generation.

协同推理扩散模型自回归多模态生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。