arXiv:2506.18251cs.GRcs.AI2025-06中稿 · ICML

Morse通过双模型协同实现扩散模型无损加速,生成速度提升1.78到3.31倍。

Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models

  • 采用快跳采样与残差反馈机制,双模型交替运行提升效率。
  • 在6个图像生成任务上平均提速1.78至3.31倍,性能零损失。
  • 适用于主流扩散模型及LCM-SDXL,适合需要快速生成的场景。

本文提出Morse,一种简单的双采样框架,可无损加速扩散模型。核心思想是利用快速跳跃采样和自适应残差反馈策略,重构从噪声到数据的迭代生成过程。Morse包含两个交互模型:Dash为预训练扩散模型,运行于跳跃采样模式,为采样效率提升提供空间;Dot模型远快于Dash,学习在跳跃点根据当前观测生成残差反馈,将噪声估计提升至接近下步的Dash模型预测,无需跳跃采样。通过时间交错地串联两者输出,Morse可在保持目标生成性能的同时显著提升整体运行效率。我们提出的参数共享策略使训练与推理均高效。在6个图像生成任务上,相对于9种基线扩散模型,平均实现1.78倍至3.31倍的无损加速。此外,该方法还可推广至已通过一致性蒸馏加速的潜空间一致性模型(LCM-SDXL),进一步优化少步数文生图生成。代码与模型见https://github.com/deep-optimization/Morse。

原文摘要 · Abstract (English)

In this paper, we present Morse, a simple dual-sampling framework for accelerating diffusion models losslessly. The key insight of Morse is to reformulate the iterative generation (from noise to data) process via taking advantage of fast jump sampling and adaptive residual feedback strategies. Specifically, Morse involves two models called Dash and Dot that interact with each other. The Dash model is just the pre-trained diffusion model of any type, but operates in a jump sampling regime, creating sufficient space for sampling efficiency improvement. The Dot model is significantly faster than the Dash model, which is learnt to generate residual feedback conditioned on the observations at the current jump sampling point on the trajectory of the Dash model, lifting the noise estimate to easily match the next-step estimate of the Dash model without jump sampling. By chaining the outputs of the Dash and Dot models run in a time-interleaved fashion, Morse exhibits the merit of flexibly attaining desired image generation performance while improving overall runtime efficiency. With our proposed weight sharing strategy between the Dash and Dot models, Morse is efficient for training and inference. Our method shows a lossless speedup of 1.78X to 3.31X on average over a wide range of sampling step budgets relative to 9 baseline diffusion models on 6 image generation tasks. Furthermore, we show that our method can be also generalized to improve the Latent Consistency Model (LCM-SDXL, which is already accelerated with consistency distillation technique) tailored for few-step text-to-image synthesis. The code and models are available at https://github.com/deep-optimization/Morse.

扩散模型无损加速双采样图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。