arXiv:2503.18940cs.CV2025-03被引 25

不重新训练,通过低分辨率先验加速扩散模型生成

Training-free Diffusion Acceleration with Bottleneck Sampling

  • 采用高-低-高去噪流程,在中间阶段降低分辨率以减少计算量
  • 图像生成提速3倍,视频生成提速2.5倍,质量与全分辨率相当
  • 无需训练,适合希望快速部署扩散模型的开发者

扩散模型在视觉内容生成方面表现出色,但推理时计算成本高,主要源于自注意力机制随图像或视频分辨率呈二次增长。现有加速方法常牺牲生成质量或需昂贵重训练。我们发现大多数扩散模型在较低分辨率下预训练,这为利用低分辨率先验实现高效推理提供了机会。本文提出Bottleneck Sampling,一种无需训练的框架,通过低分辨率先验降低计算开销并保持输出保真度。该方法采用高-低-高去噪流程:初始和末尾阶段进行高分辨率去噪,中间步骤则在低分辨率下运行。为缓解混叠和模糊伪影,我们优化了分辨率转换点,并自适应调整各阶段的去噪时间步。在图像和视频生成任务上的大量实验表明,该方法使图像生成推理速度提升最高达3倍,视频生成提升2.5倍,且在多个评估指标上保持与标准全分辨率采样相当的生成质量。

原文摘要 · Abstract (English)

Diffusion models have demonstrated remarkable capabilities in visual content generation but remain challenging to deploy due to their high computational cost during inference. This computational burden primarily arises from the quadratic complexity of self-attention with respect to image or video resolution. While existing acceleration methods often compromise output quality or necessitate costly retraining, we observe that most diffusion models are pre-trained at lower resolutions, presenting an opportunity to exploit these low-resolution priors for more efficient inference without degrading performance. In this work, we introduce Bottleneck Sampling, a training-free framework that leverages low-resolution priors to reduce computational overhead while preserving output fidelity. Bottleneck Sampling follows a high-low-high denoising workflow: it performs high-resolution denoising in the initial and final stages while operating at lower resolutions in intermediate steps. To mitigate aliasing and blurring artifacts, we further refine the resolution transition points and adaptively shift the denoising timesteps at each stage. We evaluate Bottleneck Sampling on both image and video generation tasks, where extensive experiments demonstrate that it accelerates inference by up to 3$\times$ for image generation and 2.5$\times$ for video generation, all while maintaining output quality comparable to the standard full-resolution sampling process across multiple evaluation metrics.

扩散模型加速推理无训练视频生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。