让统一模型像人一样一步步思考修正,提升多模态任务表现
UniT: Unified Multimodal Chain-of-Thought Test-time Scaling
- 通过多轮推理机制让模型逐步验证和改进输出
- 测试时扩展计算量可显著提升多模态理解与生成能力
- 适合需要复杂推理的视觉问答、内容编辑等场景
统一模型可在单一架构中处理多模态理解和生成任务,但通常仅单次完成,缺乏迭代优化。许多复杂多模态任务(如空间组合、多对象交互或动态指令)需分解指令、验证中间结果并迭代修正。尽管测试时扩展(TTS)已证明增加推理计算能显著提升语言模型性能,但将其推广至统一多模态模型仍具挑战。本文提出UniT,一种支持多模态思维链测试时扩展的框架,使单一统一模型能在多轮中实现推理、验证与精炼。该框架结合代理数据合成、统一模型训练与灵活测试时推理,激发验证、子目标分解与内容记忆等认知行为。关键发现:(1) 在短推理轨迹上训练的统一模型可泛化到更长的推理链;(2) 顺序思维链推理比并行采样更具可扩展性与计算效率;(3) 在生成与编辑轨迹上训练可提升分布外视觉推理能力。这些结果确立了多模态测试时扩展作为推动统一模型生成与理解能力的有效范式。
原文摘要 · Abstract (English)
Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs. Many multimodal tasks, especially those involving complex spatial compositions, multiple interacting objects, or evolving instructions, require decomposing instructions, verifying intermediate results, and making iterative corrections. While test-time scaling (TTS) has demonstrated that allocating additional inference compute for iterative reasoning substantially improves language model performance, extending this paradigm to unified multimodal models remains an open challenge. We introduce UniT, a framework for multimodal chain-of-thought test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds. UniT combines agentic data synthesis, unified model training, and flexible test-time inference to elicit cognitive behaviors including verification, subgoal decomposition, and content memory. Our key findings are: (1) unified models trained on short reasoning trajectories generalize to longer inference chains at test time; (2) sequential chain-of-thought reasoning provides a more scalable and compute-efficient TTS strategy than parallel sampling; (3) training on generation and editing trajectories improves out-of-distribution visual reasoning. These results establish multimodal test-time scaling as an effective paradigm for advancing both generation and understanding in unified models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。