动态调度显著提升异构GPU集群利用率与公平性。
Reducing Fragmentation and Starvation in GPU Clusters through Dynamic Multi-Objective Scheduling
- 设计三种动态调度器，兼顾利用、公平与吞吐。
- 最高实现78.2%利用率，吞吐达25.8任务/小时。
- 适合多租户AI集群部署，解决碎片化与饥饿问题。
GPU集群已成为现代AI系统训练与部署的核心，但实际部署中平均利用率仍接近50%。这种低效主要由碎片化、异构工作负载及静态调度策略局限性导致。本文系统评估了这些问题，并提出三种专用动态调度器：混合优先级（HPS）、预测性回填（PBS）和智能批处理（SBS），旨在提升多租户GPU集群的利用率、公平性和整体吞吐量。在64-GPU、8节点集群上对1,000个真实AI任务进行仿真评估，包含训练、推理和研究类负载。静态基线（FIFO、SJF、Shortest、Shortest-GPU）利用率仅45%至67%，每小时完成12.5至18.3个任务，且严重存在饥饿现象，最多有156个任务等待超30分钟。动态调度器显著优于静态策略：HPS实现最高利用率（78.2%）、最高吞吐量（25.8任务/小时），公平性方差最低（457），饥饿任务减少至12个；PBS改善碎片化处理，利用率达76.1%；SBS提升结构相似任务效率，利用率74.6%。在吞吐、等待时间、公平性方差和饥饿程度等关键指标上，动态多目标调度器全面超越单目标启发式方法。结果表明，针对性且透明的调度策略可显著提升异构AI集群的GPU效率，为未来生产调度框架提供实用基础。
原文摘要 · Abstract (English)
GPU clusters have become essential for training and deploying modern AI systems, yet real deployments continue to report average utilization near 50%. This inefficiency is largely caused by fragmentation, heterogeneous workloads, and the limitations of static scheduling policies. This work presents a systematic evaluation of these issues and introduces three specialized dynamic schedulers: Hybrid Priority (HPS), Predictive Backfill (PBS), and Smart Batch (SBS). These schedulers are designed to improve utilization, fairness, and overall throughput in multi-tenant GPU clusters. We evaluate all schedulers using a controlled simulation of 1,000 AI jobs on a 64-GPU, 8-node cluster that includes a realistic mix of training, inference, and research workloads. Static baselines (FIFO, SJF, Shortest, Shortest-GPU) achieve 45 to 67% GPU utilization and 12.5 to 18.3 jobs per hour and experience severe starvation, with as many as 156 jobs waiting longer than 30 minutes. The dynamic schedulers significantly outperform these policies. HPS achieves the highest utilization (78.2%), highest throughput (25.8 jobs per hour), and the lowest fairness variance among dynamic methods (457), reducing starvation to 12 jobs. PBS improves fragmentation handling and reaches 76.1% utilization, while SBS increases efficiency for structurally similar jobs and reaches 74.6% utilization. Across all key metrics, including throughput, job wait times, fairness variance, and starvation, dynamic multi-objective schedulers consistently outperform single-objective heuristics. These results show that targeted and transparent scheduling strategies can meaningfully increase GPU efficiency in heterogeneous AI clusters and provide a practical foundation for future production scheduling frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。