arXiv:2509.21498cs.LGcs.CV2025-09

无需训练,通过激活信息自动压缩扩散模型,速度提升35%且参数减少1亿。

SlimDiff: Training-Free, Activation-Guided Hands-free Slimming of Diffusion Models

  • 根据各阶段激活特征动态剪枝,按模块分组降低注意力与前馈维度。
  • 在不损失生成质量前提下,实现约1亿参数减少与35%推理加速。
  • 仅需500个校准样本,比之前方法少70倍,适合快速部署场景。

扩散模型(DMs)因生成效果优异而受到关注,但其百亿级参数和迭代去噪过程导致计算开销巨大。现有效率技术如量化、时间步减少或剪枝虽能节省计算、内存或运行时间,但受限于需微调或重训练以恢复性能。本文提出SlimDiff,一种完全无梯度的自动化激活引导结构压缩框架,可同时降低扩散模型中注意力与前馈层的维度。该方法将压缩重构为谱逼近问题,利用不同去噪时间步间的激活协方差定义低秩子空间,指导在固定压缩预算下的动态剪枝。通过在查询-键交互、值-输出耦合及前馈投影等功能权重组上进行模块化分解,而非孤立矩阵分解,有效缓解时间步间误差累积,并自适应分配稀疏性以契合扩散轨迹的非均匀几何结构。实验表明,SlimDiff相较基线实现最高达35%的加速与约1亿参数减少,且生成质量与未压缩模型相当,全程无需反向传播。关键在于,本方法仅需约500个校准样本,较先前方法少70倍以上。据我们所知,这是首个闭式解、激活引导的扩散模型结构压缩方法,实现全训练免费,兼具理论清晰性与实际高效性。

原文摘要 · Abstract (English)

Diffusion models (DMs), lauded for their generative performance, are computationally prohibitive due to their billion-scale parameters and iterative denoising dynamics. Existing efficiency techniques, such as quantization, timestep reduction, or pruning, offer savings in compute, memory, or runtime but are strictly bottlenecked by reliance on fine-tuning or retraining to recover performance. In this work, we introduce SlimDiff, an automated activation-informed structural compression framework that reduces both attention and feedforward dimensionalities in DMs, while being entirely gradient-free. SlimDiff reframes DM compression as a spectral approximation task, where activation covariances across denoising timesteps define low-rank subspaces that guide dynamic pruning under a fixed compression budget. This activation-aware formulation mitigates error accumulation across timesteps by applying module-wise decompositions over functional weight groups: query--key interactions, value--output couplings, and feedforward projections, rather than isolated matrix factorizations, while adaptively allocating sparsity across modules to respect the non-uniform geometry of diffusion trajectories. SlimDiff achieves up to 35\% acceleration and $\sim$100M parameter reduction over baselines, with generation quality on par with uncompressed models without any backpropagation. Crucially, our approach requires only about 500 calibration samples, over 70$\times$ fewer than prior methods. To our knowledge, this is the first closed-form, activation-guided structural compression of DMs that is entirely training-free, providing both theoretical clarity and practical efficiency.

扩散模型结构压缩无训练推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。