用连续图像标记实现大规模自回归图像生成，性能超越现有方法。
NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale
- 结合离散文本与连续图像标记，采用自回归预测训练
- 140亿参数模型在图文生成任务中达顶尖水平
- 适合追求高效高保真图像生成的研究者
当前主流的自回归文本到图像生成模型要么依赖计算密集型扩散模型处理连续图像标记，要么使用向量量化获取离散标记并引入量化损失。本文提出NextStep-1，一个140亿参数的自回归模型，搭配1.57亿参数的流匹配头，在离散文本标记与连续图像标记上以逐标记预测为目标进行训练。该模型在自回归图像生成任务中达到当前最优性能，展现出强大的高保真图像合成能力。此外，其在图像编辑任务中也表现优异，凸显了统一方法的强大与通用性。为推动开放研究，我们将公开代码与模型。
原文摘要 · Abstract (English)
Prevailing autoregressive (AR) models for text-to-image generation either rely on heavy, computationally-intensive diffusion models to process continuous image tokens, or employ vector quantization (VQ) to obtain discrete tokens with quantization loss. In this paper, we push the autoregressive paradigm forward with NextStep-1, a 14B autoregressive model paired with a 157M flow matching head, training on discrete text tokens and continuous image tokens with next-token prediction objectives. NextStep-1 achieves state-of-the-art performance for autoregressive models in text-to-image generation tasks, exhibiting strong capabilities in high-fidelity image synthesis. Furthermore, our method shows strong performance in image editing, highlighting the power and versatility of our unified approach. To facilitate open research, we will release our code and models to the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。