arXiv:2505.14741cs.LGcs.AI2025-05NeurIPS被引 2

通过复用与预测机制,实现扩散模型并行推理的高效通信。

Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism

  • 利用相邻去噪步骤的相似性,分步轻量通信实现并行化。
  • 在SVD、CogVideoX-2b和AudioLDM2-large上分别提速3.88倍、2.43倍和6.56倍。
  • 适合带宽受限场景,尤其适用于商业硬件部署。

扩散模型在图像、视频和音频生成等多模态任务中表现出强大能力,但其部署常受推理延迟制约,主要源于去噪过程的固有串行性。现有并行化方法虽能加速计算,却普遍带来高通信开销,限制了在商用硬件上的应用。为此,本文提出基于复用-预测机制的新型并行化方法ParaStep,通过挖掘相邻去噪步骤间的相似性,实现分步轻量级通信,显著降低通信开销。ParaStep在SVD上实现高达3.88倍、CogVideoX-2b上2.43倍、AudioLDM2-large上6.56倍的端到端加速,同时保持生成质量。结果表明,ParaStep是一种可扩展且通信高效的扩散模型推理加速方案,特别适用于带宽受限环境。

原文摘要 · Abstract (English)

Diffusion models have emerged as a powerful class of generative models across various modalities, including image, video, and audio synthesis. However, their deployment is often limited by significant inference latency, primarily due to the inherently sequential nature of the denoising process. While existing parallelization strategies attempt to accelerate inference by distributing computation across multiple devices, they typically incur high communication overhead, hindering deployment on commercial hardware. To address this challenge, we propose \textbf{ParaStep}, a novel parallelization method based on a reuse-then-predict mechanism that parallelizes diffusion inference by exploiting similarity between adjacent denoising steps. Unlike prior approaches that rely on layer-wise or stage-wise communication, ParaStep employs lightweight, step-wise communication, substantially reducing overhead. ParaStep achieves end-to-end speedups of up to \textbf{3.88}$\times$ on SVD, \textbf{2.43}$\times$ on CogVideoX-2b, and \textbf{6.56}$\times$ on AudioLDM2-large, while maintaining generation quality. These results highlight ParaStep as a scalable and communication-efficient solution for accelerating diffusion inference, particularly in bandwidth-constrained environments.

扩散模型并行推理通信优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。