探究多模态模型理解如何影响生成，发现推理与知识迁移存在断层。
Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward
- 构建解耦评估框架UniSandbox，用合成数据避免信息泄露。
- 显式思维链可显著缩小理解与生成差距，自训练能实现隐式推理。
- 查询架构自带类思维链特性，助力新知识迁移，适合模型优化研究者。
近年来统一多模态模型取得显著进展，但一个根本问题仍待解答：理解是否真正指导生成？为探究此问题，我们提出UniSandbox——一个解耦的评估框架，搭配受控的合成数据集，避免数据泄露并支持细致分析。研究发现理解与生成之间存在显著差距，主要体现在推理生成和知识迁移两个维度。具体而言，在推理生成任务中，理解模块中的显式思维链（CoT）有效弥合了差距，并进一步证明自训练可使模型内化该能力，实现生成时的隐式推理。在知识迁移任务中，我们发现CoT有助于生成过程中的新知识检索，还发现基于查询的架构本身具有潜在的类CoT特性，影响知识传递。UniSandbox为未来统一架构与训练策略的设计提供了初步洞见。代码与数据已公开于https://github.com/PKU-YuanGroup/UniSandBox。
原文摘要 · Abstract (English)
Recent years have witnessed significant progress in Unified Multimodal Models, yet a fundamental question remains: Does understanding truly inform generation? To investigate this, we introduce UniSandbox, a decoupled evaluation framework paired with controlled, synthetic datasets to avoid data leakage and enable detailed analysis. Our findings reveal a significant understanding-generation gap, which is mainly reflected in two key dimensions: reasoning generation and knowledge transfer. Specifically, for reasoning generation tasks, we observe that explicit Chain-of-Thought (CoT) in the understanding module effectively bridges the gap, and further demonstrate that a self-training approach can successfully internalize this ability, enabling implicit reasoning during generation. Additionally, for knowledge transfer tasks, we find that CoT assists the generative process by helping retrieve newly learned knowledge, and also discover that query-based architectures inherently exhibit latent CoT-like properties that affect this transfer. UniSandbox provides preliminary insights for designing future unified architectures and training strategies that truly bridge the gap between understanding and generation. Code and data are available at https://github.com/PKU-YuanGroup/UniSandBox
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。