提出F-scheduler，让扩散模型6步生成1024分辨率图像，性能超现有蒸馏模型。
F-scheduler: illuminating the free-lunch design space for fast sampling of diffusion models
- 设计新时间调度策略，保留部分噪声以适配变分自编码器结构。
- 在DPM++ 2m和UniPC上实现5-6步采样，1024×1024图像FID优于最先进蒸馏模型。
- 无需训练，兼容主流少步数求解器，适合高效高分辨生成任务。
扩散模型是生成高分辨率图像的领先方法，但采样计算成本高，催生了快速采样研究。尽管Free-U Net是提升图像质量的无训练增强手段，但在少步采样（<10步）下效果不佳。本文分析离散扩散微分方程，提出F-scheduler，一种专为使用Free-U Net的ODE求解器设计的调度器。该调度器包含不完全去噪特征的时间安排，使KL项能在β-VAE解码器中有效利用，并设计合适推理阶段以通过Free-U Net调整U-Net跳跃连接。基于信息论，揭示了优化调度的ODE求解器为何能超越基于训练的扩散蒸馏模型。所提调度器兼容多数少步数ODE求解器，在DPM++ 2m与UniPC下可实现1024×1024图像6步采样、512×512图像5步采样，其FID表现优于当前最优蒸馏模型及20步的DPM++ 2m求解器。代码已开源。
原文摘要 · Abstract (English)
Diffusion models are the state-of-the-art generative models for high-resolution images, but sampling from pretrained models is computationally expensive, motivating interest in fast sampling. Although Free-U Net is a training-free enhancement for improving image quality, we find it ineffective under few-step ($<10$) sampling. We analyze the discrete diffusion ODE and propose F-scheduler, a scheduler designed for ODE solvers with Free-U Net. Our proposed scheduler consists of a special time schedule that does not fully denoise the feature to enable the use of the KL-term in the $β$-VAE decoder, and the schedule of a proper inference stage for modifying the U-Net skip-connection via Free-U Net. Via information theory, we provide insights into how the better scheduled ODE solvers for the diffusion model can outperform the training-based diffusion distillation model. The newly proposed scheduler is compatible with most of the few-step ODE solvers and can sample a 1024 x 1024-resolution image in 6 steps and a 512 x 512-resolution image in 5 steps when it applies to DPM++ 2m and UniPC, with an FID result that outperforms the SOTA distillation models and the 20-step DPM++ 2m solver, respectively. Codebase: https://github.com/TheLovesOfLadyPurple/F-scheduler
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。