统一扩散架构让模型同时精通文本推理、图文理解与文生图，性能全面超越现有模型。
MMaDA: Multimodal Large Diffusion Language Models
- 采用统一扩散框架，无需模态专用模块，实现跨模态无缝融合。
- 通过混合思维链训练，使文本与视觉推理对齐，提升复杂任务初始表现。
- 提出通用强化学习算法UniGRPO，统一优化生成与推理任务，效果显著。
我们提出MMaDA，一种新型多模态扩散基础模型，旨在在文本推理、多模态理解及文生图等多元领域实现卓越性能。其核心创新包括：(i) 采用统一扩散架构与共享概率公式，具备模态无关设计，无需模态特定组件，实现不同数据类型的无缝集成与处理；(ii) 实现混合长链思维链（CoT）微调策略，统一跨模态的CoT格式，对齐文本与视觉领域的推理流程，支持最终强化学习阶段的冷启动训练，显著提升模型处理复杂任务的能力；(iii) 提出专为扩散基础模型设计的统一策略梯度强化学习算法UniGRPO，结合多样化奖励建模，统一后训练阶段在推理与生成任务上的优化，确保一致性能提升。实验表明，MMaDA-8B展现出强大的泛化能力：在文本推理上超越LLaMA-3-7B与Qwen2-7B；在多模态理解上优于Show-o与SEED-X；在文生图任务中胜过SDXL与Janus。这些成果凸显了其在统一扩散架构中弥合预训练与后训练差距的有效性，为未来研究提供完整框架。代码与模型已开源：https://github.com/Gen-Verse/MMaDA
原文摘要 · Abstract (English)
We introduce MMaDA, a novel class of multimodal diffusion foundation models designed to achieve superior performance across diverse domains such as textual reasoning, multimodal understanding, and text-to-image generation. The approach is distinguished by three key innovations: (i) MMaDA adopts a unified diffusion architecture with a shared probabilistic formulation and a modality-agnostic design, eliminating the need for modality-specific components. This architecture ensures seamless integration and processing across different data types. (ii) We implement a mixed long chain-of-thought (CoT) fine-tuning strategy that curates a unified CoT format across modalities. By aligning reasoning processes between textual and visual domains, this strategy facilitates cold-start training for the final reinforcement learning (RL) stage, thereby enhancing the model's ability to handle complex tasks from the outset. (iii) We propose UniGRPO, a unified policy-gradient-based RL algorithm specifically tailored for diffusion foundation models. Utilizing diversified reward modeling, UniGRPO unifies post-training across both reasoning and generation tasks, ensuring consistent performance improvements. Experimental results demonstrate that MMaDA-8B exhibits strong generalization capabilities as a unified multimodal foundation model. It surpasses powerful models like LLaMA-3-7B and Qwen2-7B in textual reasoning, outperforms Show-o and SEED-X in multimodal understanding, and excels over SDXL and Janus in text-to-image generation. These achievements highlight MMaDA's effectiveness in bridging the gap between pretraining and post-training within unified diffusion architectures, providing a comprehensive framework for future research and development. We open-source our code and trained models at: https://github.com/Gen-Verse/MMaDA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。