用轻重模型交替加速扩散模型采样，不降质量只减计算量。
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network
- 轻模型每步复用重模型稀疏提取的特征，减少重复计算。
- ImageNet上计算量降低2到4倍，生成质量与基线相当。
- 兼容蒸馏技术，可实现少步生成的进一步提速。
扩散模型虽生成效果顶尖，但推理时因反复调用庞大神经网络而计算开销高。本文提出双速率扩散方法，通过交替执行高容量上下文编码器（重模型）和轻量高效去噪模型（轻模型），实现采样加速。重模型稀疏运行以提取高维特征，轻模型在每一步复用这些特征，高效优化生成结果。该方法显著提升推理速度且不牺牲样本质量。在ImageNet基准测试中，双速率扩散模型性能与标准基线相当，计算成本降低2至4倍。此外，本方法与蒸馏技术（如时刻匹配蒸馏）兼容，可在少步生成场景下进一步提升效率。
原文摘要 · Abstract (English)
Diffusion models achieve state-of-the-art generative performance but suffer from high computational costs during inference due to the repeated evaluation of a heavy neural network. In this work, we propose Dual-Rate Diffusion, a method to accelerate sampling by interleaving the execution of a heavy high-capacity context encoder and a light efficient denoising model. The context encoder is evaluated sparsely to extract high-dimensional features, which are effectively reused by the light denoising model at every step to refine the sample efficiently. This approach significantly accelerates inference without compromising sample quality. On ImageNet benchmarks, Dual-Rate Diffusion matches the performance of standard baselines while reducing computational cost by a factor of $2$-$4$. Furthermore, we demonstrate that our method is compatible with distillation techniques, such as Moment Matching Distillation, enabling further efficiency gains in few-step generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。