不靠迭代去噪,用简单模型和目标快速生成高质量图像
ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling

- 用IMLE训练目标+小卷积网络,摒弃复杂设计
- 单步生成,ImageNet 256上FID达2.56,精度与召回俱佳
- 适合追求高效生成、反感复杂架构的研究者
生成模型历经从VAE/GAN到扩散模型/流匹配的演进,技术日益复杂,普遍认为需通过多次微小变换逐步将噪声分布转为数据分布。本文质疑此观点,提出极简生成建模方法ROMS-IMLE:仅保留基础训练目标与模型结构。训练目标采用隐式最大似然估计(IMLE),舍弃变分推断、对抗训练与数值积分等复杂方案;模型选用中等规模卷积网络,避开变压器结构。经审慎添加必要组件后,发现迭代去噪并非必需。最终模型为单步生成、参数高效,可在快速生成下实现高质量样本:在ImageNet 256上取得FID 2.56,并同时具备良好精度与召回率。
原文摘要 · Abstract (English)
Generative models have undergone many generations of evolution, from VAEs/GANs to diffusion/flow matching. Along the way, the underlying techniques have become more complicated and various beliefs about what drives strong empirical performance have taken hold. Due to the success of diffusion models and flow matching, one of the more common beliefs is the importance of transforming the noise distribution to the data distribution gradually through many small transformations. We ask whether this is truly necessary, and take a minimalist approach to designing a competitive generative model. We start with the bare-bones essentials, namely just a training objective and a model. We purposefully make both simple. For the training objective, we choose Implicit Maximum Likelihood Estimation (IMLE), and eschew more complicated alternatives such as variational inference, adversarial training and numerical integration. For the model, we eschew transformers and instead choose a moderately sized convolutional network. Then we judiciously added elements that are truly essential, which surprisingly do not include iterative denoising. The result is a single-step parameter-efficient generative model that produces high quality samples at fast speed: it achieves an FID of 2.56 on ImageNet 256 and simultaneously attains good precision and recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。