40亿参数统一模型,兼顾理解、生成与编辑,性能超越更大模型。
InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
- 统一建模+模块化设计,视觉表征解耦提升灵活性。
- 40亿参数下生成与编辑性能超140亿参数模型。
- 适合需要多任务统一能力的轻量化应用开发。
统一多模态模型(UMM)在保持强语义理解的同时,难以兼顾强大生成能力。本文提出轻量级40亿参数的InternVL-U,通过统一上下文建模与模态特异性模块设计,将先进的多模态大语言模型(MLLM)与基于MMDiT的视觉生成头融合。为弥合美学生成与高层智能的差距,构建了以推理为核心的高语义密度数据合成管道,利用思维链(CoT)对齐用户抽象意图与精细视觉生成细节。大量实验表明,尽管仅使用40亿参数,InternVL-U在生成与编辑任务上持续优于参数量超过3倍的基准模型(如BAGEL,140亿),同时保留强大的多模态理解与推理能力。
原文摘要 · Abstract (English)
Unified multimodal models (UMMs) that integrate understanding, reasoning, generation, and editing face inherent trade-offs between maintaining strong semantic comprehension and acquiring powerful generation capabilities. In this report, we present InternVL-U, a lightweight 4B-parameter UMM that democratizes these capabilities within a unified framework. Guided by the principles of unified contextual modeling and modality-specific modular design with decoupled visual representations, InternVL-U integrates a state-of-the-art Multimodal Large Language Model (MLLM) with a specialized MMDiT-based visual generation head. To further bridge the gap between aesthetic generation and high-level intelligence, we construct a comprehensive data synthesis pipeline targeting high-semantic-density tasks, such as text rendering and scientific reasoning, under a reasoning-centric paradigm that leverages Chain-of-Thought (CoT) to better align abstract user intent with fine-grained visual generation details. Extensive experiments demonstrate that InternVL-U achieves a superior performance - efficiency balance. Despite using only 4B parameters, it consistently outperforms unified baseline models with over 3x larger scales such as BAGEL (14B) on various generation and editing tasks, while retaining strong multimodal understanding and reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。