预训练扩散模型可跳过中间步骤加速生成,且无需改变原有训练目标。
Pretrained Diffusion Models Are Inherently Skipped-Step Samplers
- 发现预训练扩散模型天然具备跳步采样能力,通过跳过多个去噪步骤实现加速。
- 在OpenAI ADM、Stable Diffusion等模型上,采样步数大幅减少仍保持高质量生成。
- 该方法为通用加速方案,适合追求高效生成的开发者与研究者使用。
扩散模型在各类生成任务中已达到顶尖水平,但其序列化生成过程需大量步骤。现有方法如DDIM通过构建非马尔可夫扩散过程,在不改变训练目标的前提下减少采样步数。然而,原始扩散过程是否具备同等效率仍不明确。本文给出肯定答案,提出跳步采样机制,可在迭代生成中跳过多个中间去噪步骤,区别于传统逐步精修方式。关键发现是,该机制源自标准扩散模型的相同训练目标,表明通过马尔可夫方式实现加速采样是预训练扩散模型的内在属性。此外,我们结合该技术与DDIM,提出增强生成方法。在OpenAI ADM、Stable Diffusion及Open Sora等主流预训练模型上的实验表明,本方法显著减少采样步数的同时,仍能实现高质量生成。
原文摘要 · Abstract (English)
Diffusion models have been achieving state-of-the-art results across various generation tasks. However, a notable drawback is their sequential generation process, requiring long-sequence step-by-step generation. Existing methods, such as DDIM, attempt to reduce sampling steps by constructing a class of non-Markovian diffusion processes that maintain the same training objective. However, there remains a gap in understanding whether the original diffusion process can achieve the same efficiency without resorting to non-Markovian processes. In this paper, we provide a confirmative answer and introduce skipped-step sampling, a mechanism that bypasses multiple intermediate denoising steps in the iterative generation process, in contrast with the traditional step-by-step refinement of standard diffusion inference. Crucially, we demonstrate that this skipped-step sampling mechanism is derived from the same training objective as the standard diffusion model, indicating that accelerated sampling via skipped-step sampling via a Markovian way is an intrinsic property of pretrained diffusion models. Additionally, we propose an enhanced generation method by integrating our accelerated sampling technique with DDIM. Extensive experiments on popular pretrained diffusion models, including the OpenAI ADM, Stable Diffusion, and Open Sora models, show that our method achieves high-quality generation with significantly reduced sampling steps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。