针对异构多GPU环境优化扩散模型推理，降低延迟45%。
Spatio-Temporal Parallelism for Diffusion Model Inference on Heterogeneous Multi-GPU Systems
- 时空联合并行：动态分配计算任务，适配不同性能显卡。
- 推理延迟降低45%，资源利用率显著提升。
- 适合部署在真实异构硬件上的生成式AI系统。
扩散模型在图像生成中的广泛应用带来了巨大的计算开销，亟需高效的并行推理机制。然而，现有并行方案多针对同构集群，在存在硬件差异和背景负载波动的异构多GPU环境中，常因慢节点效应导致效率下降。本文提出Orchestra框架，通过细粒度的时空并行策略，均衡异构环境中的计算负载。时间维度上，采用基于计算能力的分层步数削减策略，动态剪枝较慢设备上的去噪步骤；空间维度上，实施弹性图像块并行，根据各显卡性能分配非均匀图像块。在负载不均的异构集群上进行的大量实验表明，相比当前最先进的补丁并行方法，Orchestra可将端到端延迟降低高达45%，显著提升异构显卡上的资源利用率。
原文摘要 · Abstract (English)
The widespread adoption of diffusion models for image generation necessitates efficient parallel inference to manage their substantial computational overhead. However, current parallel inference paradigms primarily target homogeneous clusters, often failing to maintain high efficiency in realistic, heterogeneous multi-GPU environments where hardware disparities and fluctuating background workloads cause severe straggler effects. This paper introduces Orchestra, a robust framework that orchestrates fine-grained parallelism across both temporal and spatial dimensions to harmonize computational loads in such settings. Temporally, Orchestra employs a novel computation-aware step allocator using a tiered step reduction strategy, intelligently pruning denoising steps on slower devices after warmup phases and execution synchronization. Spatially, Orchestra performs an elastic patch parallelism mechanism which adaptively adjusts the spatial workload intensity by assigning non-uniform image patches tailored to GPUs according to their computational capability. Extensive experiments on load-imbalanced and heterogeneous clusters validate Orchestra's efficacy in mitigating performance bottlenecks. Compared to patch parallelism, a state-of-the-art diffusion inference framework, our method reduces end-to-end latency by up to 45% and significantly boosts resource utilization on heterogeneous GPUs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。