用强化学习提升视觉语言模型的推理生成能力，解决多模态复杂任务难题。
MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO
- 通过三阶段训练构建统一多模态模型，融合扩散解码器与思维链监督微调。
- 在理解与生成基准上超越现有模型，数学推理任务表现尤为突出。
- 适合需要高阶推理生成的多模态应用，如智能创作与复杂问题求解。
当前文本到图像系统在处理多模态输入和复杂推理任务时存在局限。我们提出 MindOmni，一种通过强化学习实现推理生成的统一多模态大语言模型。该模型采用三阶段训练策略：(i) 设计包含仅解码器扩散模块的统一视觉语言模型；(ii) 使用思维链（Chain-of-Thought, CoT）指令数据进行监督微调；(iii) 提出推理生成策略优化（RGPO）算法，利用多模态反馈有效指导策略更新。实验表明，MindOmni 在理解与生成基准测试中均优于现有模型，尤其在数学推理指令任务中展现出精细推理生成能力。所有代码将公开于 https://github.com/TencentARC/MindOmni。
原文摘要 · Abstract (English)
Recent text-to-image systems face limitations in handling multimodal inputs and complex reasoning tasks. We introduce MindOmni, a unified multimodal large language model that addresses these challenges by incorporating reasoning generation through reinforcement learning. MindOmni leverages a three-phase training strategy: i) design of a unified vision language model with a decoder-only diffusion module, ii) supervised fine-tuning with Chain-of-Thought (CoT) instruction data, and iii) our proposed Reasoning Generation Policy Optimization (RGPO) algorithm, utilizing multimodal feedback to effectively guide policy updates. Experimental results demonstrate that MindOmni outperforms existing models, achieving impressive performance on both understanding and generation benchmarks, meanwhile showcasing advanced fine-grained reasoning generation capabilities, especially with mathematical reasoning instruction. All codes will be made public at https://github.com/TencentARC/MindOmni
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。