arXiv:2510.11962cs.LGcs.CV2025-10ICCV被引 8

根据预训练动态自适应剪枝,加速扩散模型采样且不降质

MosaicDiff: Training-free Structural Pruning for Diffusion Model Acceleration Reflecting Pretraining Dynamics

  • 按预训练快慢阶段动态调整剪枝强度,关键阶段更保守
  • 在DiT和SDXL上实现显著提速,质量不变,超越现有方法
  • 首次将训练动态映射到推理加速,适合追求高效生成的开发者

扩散模型虽具强大生成能力,但其预训练过程存在学习速度差异明显的阶段,此前的后训练加速研究未予关注。本文提出MosaicDiff框架,通过轨迹感知的结构剪枝,将扩散模型预训练动态与采样加速相匹配。观察发现,预训练中快速学习的中间阶段需更保守剪枝以保留关键特征,而早期和后期缓慢学习阶段可更激进剪枝。该自适应剪枝机制首次显式反映扩散模型预训练的学习速度变化,使模型内部训练动态与加速采样过程协调一致。在DiT和SDXL上的大量实验表明,本方法在不损失输出质量的前提下实现显著采样加速,大幅优于现有最先进方法,为无训练扩散模型加速提供了新视角。

原文摘要 · Abstract (English)

Diffusion models are renowned for their generative capabilities, yet their pretraining processes exhibit distinct phases of learning speed that have been entirely overlooked in prior post-training acceleration efforts in the community. In this study, we introduce a novel framework called MosaicDiff that aligns diffusion pretraining dynamics with post-training sampling acceleration via trajectory-aware structural pruning. Our approach leverages the observation that the middle, fast-learning stage of diffusion pretraining requires more conservative pruning to preserve critical model features, while the early and later, slow-learning stages benefit from a more aggressive pruning strategy. This adaptive pruning mechanism is the first to explicitly mirror the inherent learning speed variations of diffusion pretraining, thereby harmonizing the model's inner training dynamics with its accelerated sampling process. Extensive experiments on DiT and SDXL demonstrate that our method achieves significant speed-ups in sampling without compromising output quality, outperforming previous state-of-the-art methods by large margins, also providing a new viewpoint for more efficient and robust training-free diffusion acceleration.

扩散模型结构剪枝加速生成训练自由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。