arXiv:2505.23660cs.CV2025-05被引 10

用自回归模型重写扩散过程,实现图像生成新范式。

D-AR: Diffusion via Autoregressive Models

  • 将图像转为离散令牌序列,按粗到细顺序进行自回归生成。
  • 在ImageNet上用775M模型达2.09 FID,支持局部预览和布局控制。
  • 无需修改架构即可实现流式生成,适合大语言模型融合视觉合成。

本文提出D-AR,一种将图像扩散过程重新建模为标准自回归生成的新范式。通过设计将图像转化为离散令牌序列的分词器,使不同位置的令牌可解码为像素空间中不同去噪步骤的内容。由于扩散特性,这些令牌天然具有从粗到细的顺序,直接适用于自回归建模。因此,我们在不改变因果掩码或训练推理策略的前提下,采用标准的下一个令牌预测方法,生成过程自然对应图像空间中的扩散流程。一旦生成部分令牌,即可实时解码为对应的去噪步骤。该流程支持生成子集时的一致预览,并实现零样本布局控制。在标准ImageNet基准上,使用775M Llama主干与256个离散令牌,FID达到2.09。我们希望该工作能启发未来基于大语言模型的统一视觉合成自回归架构研究。代码与模型将开源。

原文摘要 · Abstract (English)

This paper presents Diffusion via Autoregressive models (D-AR), a new paradigm recasting the image diffusion process as a vanilla autoregressive procedure in the standard next-token-prediction fashion. We start by designing the tokenizer that converts images into sequences of discrete tokens, where tokens in different positions can be decoded into different diffusion denoising steps in the pixel space. Thanks to the diffusion properties, these tokens naturally follow a coarse-to-fine order, which directly lends itself to autoregressive modeling. Therefore, we apply standard next-token prediction on these tokens, without modifying any underlying designs (either causal masks or training/inference strategies), and such sequential autoregressive token generation directly mirrors the diffusion procedure in image space. That is, once the autoregressive model generates an increment of tokens, we can directly decode these tokens into the corresponding diffusion denoising step in the streaming manner. Our pipeline naturally reveals several intriguing properties, for example, it supports consistent previews when generating only a subset of tokens and enables zero-shot layout-controlled synthesis. On the standard ImageNet benchmark, our method achieves 2.09 FID using a 775M Llama backbone with 256 discrete tokens. We hope our work can inspire future research on unified autoregressive architectures of visual synthesis, especially with large language models. Code and models will be available at https://github.com/showlab/D-AR

扩散模型自回归图像生成大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。