用低分辨率逐步生成图像,大幅提升扩散模型采样速度。
LowDiff: Efficient Diffusion Sampling with Low-Resolution Condition
- 分阶段从低分辨率逐步提升生成质量,统一模型完成全过程
- 在多个数据集上实现超50%的吞吐量提升,保持或超越原有质量
- 适用于像素空间与潜在空间模型,适合需要高效生成的场景
扩散模型在图像生成中取得显著成功,但实际应用常受采样速度慢的制约。以往提升效率的方法多聚焦于压缩模型或减少去噪步数,忽略了在生成过程中利用多分辨率输入的可能性。本文提出LowDiff,一种基于级联架构的新颖高效扩散框架,通过逐步生成更高分辨率输出实现加速。同时,采用统一模型从低分辨率到目标分辨率持续优化图像。该设计使模型在大幅减少高分辨率采样步数的同时,仍能保持甚至超越原有性能。LowDiff适用于像素空间与潜在空间的扩散模型。在CIFAR-10、FFHQ和ImageNet上的大量实验表明,该方法在各类条件与无条件生成任务中均具有效性与通用性。所有设置下吞吐量提升超过50%,且生成质量相当或更优:在无条件CIFAR-10上,FID为2.11,IS为9.87;在条件CIFAR-10上,FID为1.94,IS为10.03;在FFHQ 64x64上,FID为2.43;在ImageNet 256x256上,基于LightningDiT-B/1构建的LowDiff生成样本FID为4.00,IS达195.06,效率显著提升。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable success in image generation but their practical application is often hindered by the slow sampling speed. Prior efforts of improving efficiency primarily focus on compressing models or reducing the total number of denoising steps, largely neglecting the possibility to leverage multiple input resolutions in the generation process. In this work, we propose LowDiff, a novel and efficient diffusion framework based on a cascaded approach by generating increasingly higher resolution outputs. Besides, LowDiff employs a unified model to progressively refine images from low resolution to the desired resolution. With the proposed architecture design and generation techniques, we achieve comparable or even superior performance with much fewer high-resolution sampling steps. LowDiff is applicable to diffusion models in both pixel space and latent space. Extensive experiments on both conditional and unconditional generation tasks across CIFAR-10, FFHQ and ImageNet demonstrate the effectiveness and generality of our method. Results show over 50% throughput improvement across all datasets and settings while maintaining comparable or better quality. On unconditional CIFAR-10, LowDiff achieves an FID of 2.11 and IS of 9.87, while on conditional CIFAR-10, an FID of 1.94 and IS of 10.03. On FFHQ 64x64, LowDiff achieves an FID of 2.43, and on ImageNet 256x256, LowDiff built on LightningDiT-B/1 produces high-quality samples with a FID of 4.00 and an IS of 195.06, together with substantial efficiency gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。