arXiv:2510.06590cs.CV2025-10被引 28

用连续令牌统一图像理解与生成,提升模型表现。

Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer

  • 提出连续潜空间的视觉令牌化方法MingTok,分三阶段处理
  • 在多个任务上达到顶尖性能,支持多轮交互式推理
  • 适合需要统一视觉理解与生成的AI研究者使用

视觉令牌化仍是自回归范式中统一视觉理解与生成的核心挑战。现有方法通常在离散潜空间中使用令牌化器以对齐大语言模型的令牌,但量化误差会限制语义表达力并降低视觉-语言理解能力。为此,我们提出MingTok,一种新型视觉令牌化器家族,采用连续潜空间,实现统一的自回归生成与理解。理解任务偏好高维判别特征,生成任务则倾向紧凑低层编码,MingTok通过低层编码、语义扩展和视觉重建三阶段架构调和二者矛盾。基于此,Ming-UniVision无需任务特定视觉表示,将多样视觉-语言任务统一于单一自回归预测范式。通过在共享连续空间中将理解与生成均建模为下一项预测,该模型可无缝支持多轮、上下文感知的任务,如迭代理解、生成与编辑。实验表明,使用统一连续视觉表示能有效调和理解与生成对令牌化器的冲突需求,从而在两个领域均取得当前最优性能。推理代码与模型权重已公开,以促进社区发展。

原文摘要 · Abstract (English)

Visual tokenization remains a core challenge in unifying visual understanding and generation within the autoregressive paradigm. Existing methods typically employ tokenizers in discrete latent spaces to align with the tokens from large language models, where the quantization errors can limit semantic expressiveness and degrade the capability of vision-language understanding. To address this, we introduce MingTok, a new family of visual tokenizers with a continuous latent space, for unified autoregressive generation and understanding. While understanding tasks favor discriminative high-dimensional features, generation tasks prefer compact low-level codes. Thus, to reconcile these competing demands, MingTok adopts a three-stage sequential architecture involving low-level encoding, semantic expansion, and visual reconstruction. Built on top of it, Ming-UniVision eliminates the need for task-specific visual representations, and unifies diverse vision-language tasks under a single autoregrsssive prediction paradigm. By formulating both understanding and generation as next-token prediction in a shared continuous space, it seamlessly supports multi-round, in-context tasks such as iterative understanding, generation and editing. Empirically, we find that using a unified continuous visual representation reconciles the competing requirements on the tokenizers by the understanding and generation tasks, thereby leading to state-of-the-art level performance across both domains. We hope our findings will facilitate unified visual tokenization in the continuous domain. Inference code and model weights are released to benefit community.

视觉生成连续令牌统一建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。