首个统一多模态模型代码库，支持评估与后训练。
TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training
- 构建统一接口，兼容多种多模态模型架构。
- 覆盖理解、生成、编辑三类任务，支持新旧数据集评测。
- 适合研究者快速对比模型性能，推动多模态系统发展。
近年来，统一多模态模型（UMMs）的发展催生了众多可在视觉与文本模态间进行理解、生成和编辑的模型架构。然而，由于模型架构多样、训练范式和实现细节差异大，构建统一的框架仍具挑战。本文提出TorchUMM，首个支持跨多样化UMM骨干网络、任务与数据集的全面评估、分析与后训练的统一代码库。TorchUMM涵盖广泛规模与设计范式的模型，其基准测试覆盖多模态理解、生成与编辑三大核心任务维度，整合既有与新兴数据集，用于评估感知、推理、组合性及指令遵循能力。通过提供统一接口与标准化评估协议，TorchUMM实现了异构模型间的公平、可复现比较，并促进对模型优劣的深入洞察，助力更强大统一多模态系统的开发。代码已公开：https://github.com/AIFrontierLab/TorchUMM。
原文摘要 · Abstract (English)
Recent advances in unified multimodal models (UMMs) have led to a proliferation of architectures capable of understanding, generating, and editing across visual and textual modalities. However, developing a unified framework for UMMs remains challenging due to the diversity of model architectures and the heterogeneity of training paradigms and implementation details. In this paper, we present TorchUMM, the first unified codebase for comprehensive evaluation, analysis, and post-training across diverse UMM backbones, tasks, and datasets. TorchUMM supports a broad spectrum of models covering a wide range of scales and design paradigms. Our benchmark encompasses three core task dimensions: multimodal understanding, generation, and editing, and integrates both established and novel datasets to evaluate perception, reasoning, compositionality, and instruction-following abilities. By providing a unified interface and standardized evaluation protocols, TorchUMM enables fair and reproducible comparisons across heterogeneous models and fosters deeper insights into their strengths and limitations, facilitating the development of more capable unified multimodal systems. Code is available at: https://github.com/AIFrontierLab/TorchUMM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。