用分特征生成方式提升图像生成速度与质量
ARINAR: Bi-Level Autoregressive Feature-by-Feature Generative Models
- 外层预测条件向量，内层逐特征自回归生成
- 213万参数模型在ImageNet上FID达2.75，速度是SOTA的5倍
- 适合追求高效生成的视觉生成研究者
现有自回归图像生成模型采用逐标记生成策略，即预测每个标记的概率分布并从中采样。其主要挑战在于建模高维标记的复杂分布，以往方法或过于简单无法拟合，或导致生成速度缓慢。本文提出一种分特征自回归生成思路：不再建模整个标记的分布，而是将每个标记按特征逐个生成。基于此，我们设计了双层自回归模型ARINAR（AR-in-AR），外层以先前标记为输入，预测下一个标记的条件向量z；内层在z条件下，自回归生成该标记的各个特征。这样，内层只需建模单个特征的分布，例如使用简单的高斯混合模型。在ImageNet 256x256图像生成任务中，参数量为213M的ARINAR-B模型达到FID 2.75，与当前最优模型MAR-B（FID=2.31）相当，但速度提升五倍。
原文摘要 · Abstract (English)
Existing autoregressive (AR) image generative models use a token-by-token generation schema. That is, they predict a per-token probability distribution and sample the next token from that distribution. The main challenge is how to model the complex distribution of high-dimensional tokens. Previous methods either are too simplistic to fit the distribution or result in slow generation speed. Instead of fitting the distribution of the whole tokens, we explore using a AR model to generate each token in a feature-by-feature way, i.e., taking the generated features as input and generating the next feature. Based on that, we propose ARINAR (AR-in-AR), a bi-level AR model. The outer AR layer take previous tokens as input, predicts a condition vector z for the next token. The inner layer, conditional on z, generates features of the next token autoregressively. In this way, the inner layer only needs to model the distribution of a single feature, for example, using a simple Gaussian Mixture Model. On the ImageNet 256x256 image generation task, ARINAR-B with 213M parameters achieves an FID of 2.75, which is comparable to the state-of-the-art MAR-B model (FID=2.31), while five times faster than the latter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。