用机器学习动态分配异构集群算力,边运行边优化节能与性能。
GOGH: Correlation-Guided Orchestration of GPUs in Heterogeneous Clusters
- 双神经网络在线估计算力匹配度和模型共存影响
- 实测能耗降低18%,延迟满足95%请求的SLA要求
- 适合运维复杂、设备混杂的深度学习集群
机器学习对算力需求激增,使异构硬件集群中的资源高效分配成为关键挑战。由于升级最新硬件常不可行,可持续利用现有混合代际资源尤为重要。本文提出一种基于学习的架构,用于管理异构集群中的机器学习工作负载。系统在线运行,根据性能要求和能耗约束,为新到达的训练或推理请求分配资源。采用两个神经网络:首个网络提供新模型在不同硬件上的利用率及对共驻模型的影响初步估计;优化器据此进行资源分配。部署后,系统监控实际表现,并通过第二个神经网络利用反馈数据改进预测。该更新模型不仅能提升当前硬件的预测精度,还能泛化至未分配硬件及未观测的共驻场景。最终实现持续迭代的自适应资源调度,在异构深度学习集群中提升决策效率。
原文摘要 · Abstract (English)
The growing demand for computational resources in machine learning has made efficient resource allocation a critical challenge, especially in heterogeneous hardware clusters where devices vary in capability, age, and energy efficiency. Upgrading to the latest hardware is often infeasible, making sustainable use of existing, mixed-generation resources essential. In this paper, we propose a learning-based architecture for managing machine learning workloads in heterogeneous clusters. The system operates online, allocating resources to incoming training or inference requests while minimizing energy consumption and meeting performance requirements. It uses two neural networks: the first provides initial estimates of how well a new model will utilize different hardware types and how it will affect co-located models. An optimizer then allocates resources based on these estimates. After deployment, the system monitors real performance and uses this data to refine its predictions via a second neural network. This updated model improves estimates not only for the current hardware but also for hardware not initially allocated and for co-location scenarios not yet observed. The result is an adaptive, iterative approach that learns over time to make more effective resource allocation decisions in heterogeneous deep learning clusters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。