用简单尺度设计提升图像生成质量,支持任意VAE
FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching
- 采用倍增尺度设计,简化自回归图像生成的结构
- 在ImageNet-256上生成质量超越已有方法
- 兼容任意现成VAE,提升模型模块灵活性
自回归建模在自然语言处理中通过预测下一个词元实现连贯文本生成。近期,VAR提出基于尺度的自回归建模,将词元预测扩展为尺度预测,保留图像二维结构。但VAR存在两大挑战:(1) 复杂且僵化的尺度设计限制了尺度预测的泛化能力;(2) 生成器依赖同结构的离散分词器,阻碍分词器的模块化更新。为此,我们提出FlowAR,一种简洁的尺度预测方法,每个后续尺度均为前一尺度的两倍。该设计消除VAR中复杂的多尺度残差分词器,可适配任意现成变分自编码器(VAE)。简化结构提升了尺度预测的泛化性,并支持引入流匹配(Flow Matching)以实现高质量图像合成。我们在具有挑战性的ImageNet-256基准上验证了FlowAR的有效性,生成性能优于此前方法。
原文摘要 · Abstract (English)
Autoregressive (AR) modeling has achieved remarkable success in natural language processing by enabling models to generate text with coherence and contextual understanding through next token prediction. Recently, in image generation, VAR proposes scale-wise autoregressive modeling, which extends the next token prediction to the next scale prediction, preserving the 2D structure of images. However, VAR encounters two primary challenges: (1) its complex and rigid scale design limits generalization in next scale prediction, and (2) the generator's dependence on a discrete tokenizer with the same complex scale structure restricts modularity and flexibility in updating the tokenizer. To address these limitations, we introduce FlowAR, a general next scale prediction method featuring a streamlined scale design, where each subsequent scale is simply double the previous one. This eliminates the need for VAR's intricate multi-scale residual tokenizer and enables the use of any off-the-shelf Variational AutoEncoder (VAE). Our simplified design enhances generalization in next scale prediction and facilitates the integration of Flow Matching for high-quality image synthesis. We validate the effectiveness of FlowAR on the challenging ImageNet-256 benchmark, demonstrating superior generation performance compared to previous methods. Codes will be available at \url{https://github.com/OliverRensu/FlowAR}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。