STARFlow用Transformer增强生成流,实现高分辨率图像高质量合成。
STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
- 结合Transformer与生成流,构建可扩展的自回归建模结构。
- 在512×512图像上生成质量接近扩散模型,无需离散化训练。
- 适合追求精确似然与高质量图像生成的研究者使用。
我们提出STARFlow,一种基于生成流的可扩展生成模型,在高分辨率图像合成上表现优异。其核心是Transformer自回归流(TARFlow),融合了生成流的表达力与自回归Transformer的结构建模能力。首先理论上证明了TARFlow对连续分布建模的普适性。在此基础上,引入三项关键创新:(1) 深-浅混合设计,深层Transformer承担主要表征能力,少量浅层块提升计算效率;(2) 在预训练自编码器的隐空间建模,优于直接像素级建模;(3) 一种新型引导算法显著提升样本质量。模型保持端到端生成流结构,可在连续空间中进行精确最大似然训练,无需离散化。在类别条件与文本条件图像生成任务中均表现良好,生成质量接近当前最优扩散模型。据我们所知,这是首个在该规模和分辨率下成功运行的生成流模型。
原文摘要 · Abstract (English)
We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the expressive power of normalizing flows with the structured modeling capabilities of Autoregressive Transformers. We first establish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce several key architectural and algorithmic innovations to significantly enhance scalability: (1) a deep-shallow design, wherein a deep Transformer block captures most of the model representational capacity, complemented by a few shallow Transformer blocks that are computationally efficient yet substantially beneficial; (2) modeling in the latent space of pretrained autoencoders, which proves more effective than direct pixel-level modeling; and (3) a novel guidance algorithm that significantly boosts sample quality. Crucially, our model remains an end-to-end normalizing flow, enabling exact maximum likelihood training in continuous spaces without discretization. STARFlow achieves competitive performance in both class-conditional and text-conditional image generation tasks, approaching state-of-the-art diffusion models in sample quality. To our knowledge, this work is the first successful demonstration of normalizing flows operating effectively at this scale and resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。