arXiv:2604.02158cs.DCcs.LG2026-04

基于日志与性能数据,精准预测异构超算中GPU资源与功耗。

A Practical Two-Stage Framework for GPU Resource and Power Prediction in Heterogeneous HPC Systems

  • 分两阶段建模:仅用作业日志,再融合GPU实时监控数据
  • 仅用日志即可达97%的GPU利用率预测准确率
  • 适合需要节能调度的高性能计算系统运维人员

随着高性能计算(HPC)对GPU需求增长,高效利用GPU资源与功耗日益关键。本文基于Slurm工作负载历史日志及NVIDIA DCGM采集的GPU性能指标,分析了在NERSC Perlmutter系统(基于NVIDIA A100 GPU的HPE Cray EX架构)上广泛使用的材料科学软件VASP的GPU利用率、内存利用率和功耗。基于对VASP应用资源使用特征的洞察,提出一种两阶段资源预测框架,用于预测异构HPC系统应用的平均GPU功耗、最大GPU利用率和最大GPU内存利用率,以支持更高效的调度决策与功耗感知系统运行。第一阶段仅使用Slurm记账日志作为训练数据,第二阶段加入历史DCGM性能指标进行增强。仅用提交特征预测最大GPU利用率最高可达97%准确率;从GPU计算与内存活动指标中提取的特征与平均功耗高度相关,运行时功耗预测准确率最高达92%。结果表明DCGM指标能有效捕捉应用特征,具有支撑动态功耗管理预测模型的潜力。

原文摘要 · Abstract (English)

Efficient utilization of GPU resources and power has become critical with the growing demand for GPUs in high-performance computing (HPC). In this paper, we analyze GPU utilization and GPU memory utilization, as well as the power consumption of the Vienna ab initio Simulation Package (VASP), using the Slurm workload manager historical logs and GPU performance metrics collected by NVIDIA's Data Center GPU Manager (DCGM). VASP is a widely used materials science application on Perlmutter at NERSC, an HPE Cray EX system based on NVIDIA A100 GPUs. Using our insights from the resource utilization analysis of VASP applications, we propose a resource prediction framework to predict the average GPU power, maximum GPU utilization, and maximum GPU memory utilization values of heterogeneous HPC system applications to enable more efficient scheduling decisions and power-aware system operation. Our prediction framework consists of two stages: 1) using only the Slurm accounting logs as training data and 2) augmenting the training data with historical GPU profiling metrics collected with DCGM. The maximum GPU utilization predictions using only the Slurm submission features achieve up to 97% accuracy. Furthermore, features engineered from GPU-compute and memory activity metrics exhibit good correlations with average power utilization, and our runtime power usage prediction experiments result in up to 92% prediction accuracy. These findings demonstrate the effectiveness of DCGM metrics in capturing application characteristics and highlight their potential for developing predictive models to support dynamic power management in HPC systems.

GPU调度功耗预测异构系统性能监控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。