arXiv:2607.24157cs.CV2026-07

用自回归模型统一图像生成,速度提升19倍且保持高质量。

UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling

论文配图:UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
图 1 · 摘自论文原文
  • 用多模态语言模型+自回归解码器,统一处理15种图像任务。
  • 相比扩散模型,推理延迟降低19倍,输出质量不降反升。
  • 适合需要高效图像生成的工业部署和多任务系统设计者。

现代计算机视觉系统任务分散,文本到图像生成、编辑、修复和传统感知等由不同模型处理。本文研究统一视觉生成(UVG),即通过统一的多模态接口,用单一模型生成多样图像输出。尽管基于扩散模型的系统在质量与可控性上占优,但其迭代采样导致显著推理延迟,限制实际应用。为此,我们提出UniGen-AR框架,结合通用多模态语言模型(MLLM)与高效的下一尺度视觉自回归(VAR)解码器。该设计保留MLLM的灵活条件控制能力,同时利用VAR模型的采样效率与潜在空间统一特性。在该框架中,MLLM将自由形式指令与控制信号编码为统一序列,指导VAR解码器生成超过15项任务的图像输出,覆盖四大类别。实验表明,UniGen-AR相比扩散基线推理延迟降低高达19倍,同时保持或提升输出质量。消融研究进一步揭示,VQ-VAE分词器设计(尤其是代码本大小与层级结构)是影响VAR在UVG中可扩展性的关键因素。结果表明,视觉自回归建模是统一视觉生成的高效可靠基础。项目页面见 https://zpbao.github.io/projects/unigenar。

原文摘要 · Abstract (English)

Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface. While diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.

图像生成自回归多模态高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。