用学习到的先验分布加速扩散生成,推理更快质量更高
LeDiFlow: Learned Distribution-guided Flow Matching to Accelerate Image Generation
- 用回归模型学更好先验,让生成路径更直,减少求解步数
- 像素级生成快3.75倍,潜在空间生成在CMMD上提升1.32倍
- 可在消费级设备训练,适合追求高效高质图像生成的研究者
由于扩散模型(DMs)的迭代特性,提升高质量图像生成效率是一个重大挑战。流匹配(FM)作为一种新兴生成建模范式,基于无需模拟的训练目标,而非DMs所用的基于得分的方法。典型FM方法依赖高斯先验分布,导致先验与目标数据分布间概率路径呈曲线状,给常微分方程(ODE)求解器带来困难,需大量调用流预测网络。为此,我们提出学习型分布引导的流匹配(LeDiFlow),一种基于回归辅助模型学习更合适先验分布的新方法,以实现可扩展的FM图像生成训练。通过使用更接近目标数据分布的先验初始化ODE求解器,LeDiFlow使学习到的概率路径更具计算可处理性,直接转化为推理时更少的求解步数。该方法结合当前最先进的变压器架构与潜在空间采样,可在消费级工作站上训练。实验证明,LeDiFlow显著优于相应基线:在像素空间直接运行时,推理速度比对应基线最高提升3.75倍;同时,其潜在空间模型在CLIP最大均值差异(CMMD)指标上平均提升1.32倍。
原文摘要 · Abstract (English)
Enhancing the efficiency of high-quality image generation using Diffusion Models (DMs) is a significant challenge due to the iterative nature of the process. Flow Matching (FM) is emerging as a powerful generative modeling paradigm based on a simulation-free training objective instead of a score-based one used in DMs. Typical FM approaches rely on a Gaussian distribution prior, which induces curved, conditional probability paths between the prior and target data distribution. These curved paths pose a challenge for the Ordinary Differential Equation (ODE) solver, requiring a large number of inference calls to the flow prediction network. To address this issue, we present Learned Distribution-guided Flow Matching (LeDiFlow), a novel scalable method for training FM-based image generation models using a better-suited prior distribution learned via a regression-based auxiliary model. By initializing the ODE solver with a prior closer to the target data distribution, LeDiFlow enables the learning of more computationally tractable probability paths. These paths directly translate to fewer solver steps needed for high-quality image generation at inference time. Our method utilizes a State-Of-The-Art (SOTA) transformer architecture combined with latent space sampling and can be trained on a consumer workstation. We empirically demonstrate that LeDiFlow remarkably outperforms the respective FM baselines. For instance, when operating directly on pixels, our model accelerates inference by up to 3.75x compared to the corresponding pixel-space baseline. Simultaneously, our latent FM model enhances image quality on average by 1.32x in CLIP Maximum Mean Discrepancy (CMMD) metric against its respective baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。