提出单步生成新框架，突破扩散模型效率瓶颈。
SoFlow: Solution Flow Models for One-Step Generative Modeling
- 基于速度函数与解函数关系设计新型训练损失
- 单步生成在ImageNet上超越均值流模型性能
- 无需计算雅可比向量积，适合主流深度学习框架
扩散模型和流匹配模型的多步去噪过程带来严重效率问题，推动了少步生成的研究。本文提出从零开始的单步生成框架——Solution Flow Models（SoFlow）。通过分析速度函数与速度常微分方程解函数的关系，我们设计了流匹配损失和解一致性损失用于模型训练。流匹配损失使模型在训练中能提供分类器无关引导所需的估计速度场，提升生成质量。值得注意的是，我们的一致性损失无需计算雅可比向量积（JVP），避免了当前多数工作依赖但深度学习框架优化不佳的JVP计算。实验表明，在使用相同扩散Transformer（DiT）架构并经过相同训练轮数条件下，我们的模型在ImageNet 256x256数据集上取得优于均值流模型（MeanFlow）的FID-50K得分。
原文摘要 · Abstract (English)
The multi-step denoising process in diffusion and Flow Matching models causes major efficiency issues, which motivates research on few-step generation. We present Solution Flow Models (SoFlow), a framework for one-step generation from scratch. By analyzing the relationship between the velocity function and the solution function of the velocity ordinary differential equation (ODE), we propose a Flow Matching loss and a solution consistency loss to train our models. The Flow Matching loss allows our models to provide estimated velocity fields for Classifier-Free Guidance (CFG) during training, which improves generation performance. Notably, our consistency loss does not require the calculation of the Jacobian-vector product (JVP), a common requirement in recent works that is not well-optimized in deep learning frameworks like PyTorch. Experimental results indicate that, when trained from scratch using the same Diffusion Transformer (DiT) architecture and an equal number of training epochs, our models achieve better FID-50K scores than MeanFlow models on the ImageNet 256x256 dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。