用Vision Transformer重构流模型，实现更优生成效果。
Jet: A Modern Transformer-Based Normalizing Flow
- 改用ViT结构替代卷积，提升建模能力
- 在图像生成上达到当前最优定量与定性表现
- 适合想构建高效生成模型的研究者
以往归一化流模型因其可高效计算输入数据的对数似然、生成速度快和结构简单等优势，成为自然图像生成的有力候选。尽管该类模型持续受到关注，但因生成样本视觉质量不及GAN、VQ-VAE或扩散模型而逐渐失宠。本文通过系统消融分析，重新审视基于耦合的归一化流设计，采用基于视觉变换器（Vision Transformer）的计算模块取代传统卷积神经网络。结果表明，所提方法在保持极简架构的同时，实现了当前最优的定量与定性生成性能。尽管整体视觉质量仍略逊于最先进模型，我们主张强大的归一化流模型可作为更强大生成模型的重要组件，推动研究前沿发展。
原文摘要 · Abstract (English)
In the past, normalizing generative flows have emerged as a promising class of generative models for natural images. This type of model has many modeling advantages: the ability to efficiently compute log-likelihood of the input data, fast generation and simple overall structure. Normalizing flows remained a topic of active research but later fell out of favor, as visual quality of the samples was not competitive with other model classes, such as GANs, VQ-VAE-based approaches or diffusion models. In this paper we revisit the design of the coupling-based normalizing flow models by carefully ablating prior design choices and using computational blocks based on the Vision Transformer architecture, not convolutional neural networks. As a result, we achieve state-of-the-art quantitative and qualitative performance with a much simpler architecture. While the overall visual quality is still behind the current state-of-the-art models, we argue that strong normalizing flow models can help advancing research frontier by serving as building components of more powerful generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。