动态调整生成步数并行度，提升DiT模型服务效率。
TetriServe: Efficiently Serving Mixed DiT Workloads
- 按生成步长动态调节并行度，适应不同请求的截止时间。
- 在不降低图像质量前提下，SLO达成率最高提升32%。
- 适合高并发、多分辨率图像生成服务场景使用。
扩散变压器（DiT）模型通过迭代去噪生成高质量图像，但在严格的服务水平目标（SLO）下部署面临挑战，尤其在大分辨率时计算成本高昂。现有服务系统采用固定程度的序列并行，难以应对混合分辨率和截止时间的异构工作负载，导致GPU利用率低、SLO达成率差。本文提出步级序列并行，根据每个请求的截止时间动态调整并行度。我们设计了TetriServe系统，实现该策略以高效生成图像。具体而言，TetriServe引入一种基于轮次的调度机制：（1）将时间离散为固定轮次，使带截止时间的调度可处理；（2）在步级层面调整并行度，最小化GPU小时消耗；（3）联合打包请求，减少延迟完成。在主流DiT模型上的大量评估表明，TetriServe相比现有方案在不降低图像质量的前提下，最多提升32%的SLO达成率。
原文摘要 · Abstract (English)
Diffusion Transformer (DiT) models excel at generating high-quality images through iterative denoising steps, but serving them under strict Service Level Objectives (SLOs) is challenging due to their high computational cost, particularly at larger resolutions. Existing serving systems use fixed-degree sequence parallelism, which is inefficient for heterogeneous workloads with mixed resolutions and deadlines, leading to poor GPU utilization and low SLO attainment. In this paper, we propose step-level sequence parallelism to dynamically adjust the degree of parallelism of individual requests according to their deadlines. We present TetriServe, a DiT serving system that implements this strategy for highly efficient image generation. Specifically, TetriServe introduces a novel round-based scheduling mechanism that improves SLO attainment by (1) discretizing time into fixed rounds to make deadline-aware scheduling tractable, (2) adapting parallelism at the step level and minimizing GPU hour consumption, and (3) jointly packing requests to minimize late completions. Extensive evaluation on state-of-the-art DiT models shows that TetriServe achieves up to 32% higher SLO attainment compared to existing solutions without degrading image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。