用双骨干网络加速扩散模型生成，初期浅网快出，后期深网保质。
DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach
- 前期用浅网络快速去噪，后期切换到深网络精细处理。
- 在相同速度下生成质量更高，比现有方法快且效果更好。
- 实现简单，可与现有加速方法配合使用，适合部署优化。
扩散模型在图像生成上表现卓越，但因迭代采样过程缓慢而受限。为解决此问题，早期退出机制被提出，根据每步去噪难度自适应调整网络深度。我们发现当前自适应扩散模型的采样过程中存在一种有趣的“相变”现象：去噪网络在初始步骤始终提前退出，直到某一点突然切换至完整网络。基于此，我们提出双骨干方法DuoDiff：在初始采样阶段使用浅层去噪网络，在后期使用深层网络。实验证明，DuoDiff在推理速度和生成质量上均优于现有早期退出方法。重要的是，该方法实现简单，可与现有加速技术互补，适用于实际部署中的效率优化。
原文摘要 · Abstract (English)
Diffusion models have achieved unprecedented performance in image generation, yet they suffer from slow inference due to their iterative sampling process. To address this, early-exiting has recently been proposed, where the depth of the denoising network is made adaptive based on the (estimated) difficulty of each sampling step. Here, we discover an interesting "phase transition" in the sampling process of current adaptive diffusion models: the denoising network consistently exits early during the initial sampling steps, until it suddenly switches to utilizing the full network. Based on this, we propose accelerating generation by employing a shallower denoising network in the initial sampling steps and a deeper network in the later steps. We demonstrate empirically that our dual-backbone approach, DuoDiff, outperforms existing early-exit diffusion methods in both inference speed and generation quality. Importantly, DuoDiff is easy to implement and complementary to existing approaches for accelerating diffusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。