通过梯度估计算法实现扩散模型云边协同加速,兼顾生成质量与推理速度。
EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Models
- 用梯度噪声估计减少云端推理次数,周期性调整误差
- 实现平均2倍于纯云端的推理速度提升,生成质量更优
- 适合资源受限设备部署高阶扩散模型生成任务
扩散模型在图像与视频生成方面表现出色。随着模型规模和延迟增大,用户体验受限,因此提出云边协同框架以实现快速推理与高质量生成:云端负责高质量语义规划,边缘端加速后期优化。然而,过多云端去噪会延长推理时间,而步骤不足则导致语义模糊,引发边缘输出不一致。为此,我们提出EC-Diff,通过基于梯度的噪声估计加速云端推理,并确定最优云边交接点以保持生成质量。具体地,设计了K步噪声近似策略,利用步间噪声梯度并周期性进行云端推理以修正误差;同时设计两阶段贪婪搜索算法,高效找到噪声近似与边缘切换的最优参数。大量实验表明,相比仅边缘推理,本方法显著提升生成质量,且相比纯云端推理平均提速达2倍。视频样例与源代码见https://ec-diff.github.io/。
原文摘要 · Abstract (English)
Diffusion Models have shown remarkable proficiency in image and video synthesis. As model size and latency increase limit user experience, hybrid edge-cloud collaborative framework was recently proposed to realize fast inference and high-quality generation, where the cloud model initiates high-quality semantic planning and the edge model expedites later-stage refinement. However, excessive cloud denoising prolongs inference time, while insufficient steps cause semantic ambiguity, leading to inconsistency in edge model output. To address these challenges, we propose EC-Diff that accelerates cloud inference through gradient-based noise estimation while identifying the optimal point for cloud-edge handoff to maintain generation quality. Specifically, we design a K-step noise approximation strategy to reduce cloud inference frequency by using noise gradients between steps and applying cloud inference periodically to adjust errors. Then we design a two-stage greedy search algorithm to efficiently find the optimal parameters for noise approximation and edge model switching. Extensive experiments demonstrate that our method significantly enhances generation quality compared to edge inference, while achieving up to an average $2\times$ speedup in inference compared to cloud inference. Video samples and source code are available at https://ec-diff.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。