EMMA用统一架构高效实现多模态理解、生成与编辑。
EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture
- 32倍压缩比自编码器减少令牌数，平衡理解与生成训练。
- 通道拼接替代令牌拼接，进一步降低视觉令牌数量。
- 共享解耦网络+专家混合机制，兼顾性能与效率，适合多任务场景。
我们提出EMMA，一种高效且统一的多模态理解、生成与编辑架构。EMMA主要包含：1）一个32倍压缩比的高效自编码器，显著减少生成所需令牌数，并通过相同压缩比应用于图像，确保理解与生成任务的训练平衡；2）采用通道级拼接而非令牌级拼接，进一步降低统一架构中的视觉令牌数量；3）共享-解耦网络，实现跨任务相互提升的同时满足任务特定建模需求；4）在视觉理解编码器中引入专家混合机制，仅增加少量参数即大幅提升感知能力。大量实验表明，EMMA-4B在效率和性能上均显著优于当前最先进的统一多模态方法（如BAGEL-7B），同时在多模态理解与生成方面达到与Qwen3-VL和Qwen-Image等专用模型相当的竞争力。我们认为，EMMA为未来统一多模态架构的发展奠定了坚实基础。
原文摘要 · Abstract (English)
We propose EMMA, an efficient and unified architecture for multimodal understanding, generation and editing. Specifically, EMMA primarily consists of 1) An efficient autoencoder with a 32x compression ratio, which significantly reduces the number of tokens required for generation. This also ensures the training balance between understanding and generation tasks by applying the same compression ratio to images. 2) Channel-wise concatenation instead of token-wise concatenation among visual understanding and generation tokens, which further reduces the visual tokens in unified architectures. 3) A shared-and-decoupled network that enables mutual improvements across tasks while meeting the task-specific modeling requirements. 4) A mixture-of-experts mechanism adopted for visual understanding encoder, which substantially improves perceptual capabilities with a few parameters increase. Extensive experiments have shown that EMMA-4B can significantly outperform state-of-the-art unified multimodal approaches (e.g., BAGEL-7B) in both efficiency and performance, while also achieving competitive results compared to recent multimodal understanding and generation experts (e.g., Qwen3-VL and Qwen-Image). We believe that EMMA lays a solid foundation for the future development of unified multimodal architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。