arXiv:2606.18249cs.CV2026-06被引 2

一个视觉分词器打通图文理解与生成,实现统一建模。

Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification

论文配图:Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification
图 1 · 摘自论文原文
  • 用单一离散视觉分词器连接理解与生成任务
  • 多层级特征融合+无查找位量化,提升词汇量且不增加开销
  • 并行位预测加速生成,支持高质量图像合成与编辑

统一多模态建模旨在将视觉理解与生成整合于单一系统中。然而现有方法通常依赖两个独立的视觉分词器,导致表示空间割裂,难以实现真正统一。我们提出UniAR,一种统一的自回归框架,其中单一离散视觉分词器作为理解与生成之间的关键桥梁,使模型能直接解析自身生成的视觉标记,无需额外重编码。UniAR通过多层级特征融合和无查找位量化方案适配预训练视觉编码器,在最小成本下保留高层语义与低层细节,扩展有效视觉词汇量。基于此,统一自回归模型采用并行位预测机制,联合预测空间分组的多层级视觉代码,显著缩短视觉序列长度并加速生成。最后,基于扩散的视觉解码器在离散视觉标记上生成高保真图像。经过大规模预训练、监督微调及强化学习,UniAR在图像生成与图像编辑任务上达到业界领先水平,同时在多模态理解基准上保持竞争力。

原文摘要 · Abstract (English)

Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.

多模态自回归视觉分词器图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。