提出SynerDiff系统,实现扩散模型推理的高吞吐低延迟。
SynerDiff: Synergetic Continuous Batching for Fast and Parallel Diffusion Model Inference

- 通过VAE分块和自适应跳过CFG缓解组件资源竞争。
- 调度器根据敏感度差异优化并发,使平均与尾部延迟降低78.7%。
- 适合需要高并发、低延迟生成服务的研发人员参考。
人工智能生成内容服务的扩展要求扩散模型推理在保证高吞吐的同时实现低任务端到端(E2E)延迟。然而,现有连续批处理方法在UNet-VAE并发时面临严重资源争用,导致延迟突增。此外,多任务并发调度在不同策略下存在UNet吞吐与VAE延迟的权衡。为此,我们提出SynerDiff,一种基于内-间协同机制的高效连续批处理系统。在内部并发层面,通过VAE分块和自适应跳过CFG缓解组件特定资源瓶颈;在外部并发层面,利用组件对调度粒度的差异敏感性,设计阈值感知调度器规划并发序列并调优内部并发决策,在保持UNet高吞吐的同时最小化VAE延迟。此外,反馈控制器根据队列负载动态调整该阈值,提升系统容量上限。实验表明,相比基准方法,SynerDiff将吞吐提升1.6倍,平均与P99尾部延迟最高降低78.7%,同时保障高质量图像输出。
原文摘要 · Abstract (English)
The expansion of Artificial Intelligence-generated content service requires diffusion model serving to simultaneously achieve high throughput and low task end-to-end (E2E) latency. However, existing continuous batching methods suffer from severe resource contention during UNet-VAE concurrency, leading to latency spikes. Furthermore, concurrent multi-task scheduling entails a trade-off between UNet throughput and VAE latency across varying scheduling strategies. To address these, we propose SynerDiff, an efficient continuous batching system built on intra-inter level synergy. At the intra-concurrency level, SynerDiff alleviates resource contention by pruning component-specific resource bottlenecks via VAE Chunking and Adaptive Skip-CFG. At the inter-concurrency level, leveraging components' differential sensitivity to scheduling granularities, a threshold-aware scheduler plans concurrent sequences and tunes intra-concurrency decisions to minimize VAE latency while maintaining UNet within high-throughput threshold. Additionally, a feedback controller dynamically adjusts this threshold based on queue loads to boost system capacity ceiling. Experimental results show that, SynerDiff improves throughput by 1.6$\times$ and decreases both average E2E and P99 tail latencies by up to 78.7\%, compared to benchmarks while guaranteeing high image fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。