arXiv:2410.08159cs.CVcs.LG2024-10中稿 · ICLR被引 34

用非马尔可夫框架统一自回归与扩散模型,提升文生图生成效率。

DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation

论文配图:DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation
图 1 · 摘自论文原文
  • 基于Transformer的非马尔可夫框架,分块空间-频域迭代去噪
  • 无需图像量化,在文本/图像联合训练中实现高保真生成
  • 适合追求高效、高质量图像生成的研究者与开发者

扩散模型已成为视觉生成的主流方法,通过逐步加噪的马尔可夫过程进行训练。我们指出,马尔可夫性质限制了模型对生成轨迹的充分利用,导致训练与推理效率低下。本文提出DART——一种基于Transformer的非马尔可夫框架,将自回归(AR)与扩散模型统一。DART采用与标准语言模型同构的AR结构,空间与频域上分块迭代去噪。该方法无需图像量化,提升了建模能力并保持灵活性。此外,DART可无缝联合训练文本与图像数据。在类别条件与文生图任务中表现优异,为可扩展、高效的图像生成提供了新范式,树立了高质量图像合成的新基准。

原文摘要 · Abstract (English)

Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that the Markovian property limits the model's ability to fully utilize the generation trajectory, leading to inefficiencies during training and inference. In this paper, we propose DART, a transformer-based model that unifies autoregressive (AR) and diffusion within a non-Markovian framework. DART iteratively denoises image patches spatially and spectrally using an AR model that has the same architecture as standard language models. DART does not rely on image quantization, which enables more effective image modeling while maintaining flexibility. Furthermore, DART seamlessly trains with both text and image data in a unified model. Our approach demonstrates competitive performance on class-conditioned and text-to-image generation tasks, offering a scalable, efficient alternative to traditional diffusion models. Through this unified framework, DART sets a new benchmark for scalable, high-quality image synthesis.

文生图扩散模型自回归Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。