用渐进式训练提升智能体能效,让超算集群省电又不耽误任务。
Improving the Efficiency of a Deep Reinforcement Learning-Based Power Management System for HPC Clusters Using Curriculum Learning
- 通过由易到难的课程学习训练强化学习智能体
- 比基线方法省电3.73%,任务等待时间减少9.24%
- 适应不同硬件配置,无需重新训练
高性能计算(HPC)系统能耗高,即使在空闲时也消耗大量电力。关机闲置节点可节能,但时机不当会延迟任务执行。本文采用深度强化学习(DRL)优化开关机策略,并引入课程学习(CL),按难度递增顺序训练智能体。基于Batsim-py仿真框架,对比了无CL的基准DRL方法和固定超时关机策略。结果表明,由易到难的课程学习最优:相较基线方法降低3.73%能耗,比最佳超时配置(每15分钟关机一次)节能4.66%;平均任务等待时间减少9.24%,作业填充率更高,资源利用率更好。在不同开机时间、功耗水平和集群规模下的敏感性测试显示,该智能体无需重训即可适应变化。实验证明,课程学习显著提升DRL在HPC能效管理中的表现,兼顾节能、服务质量与配置鲁棒性。
原文摘要 · Abstract (English)
High energy consumption remains a key challenge in high-performance computing (HPC) systems, which often feature hundreds or thousands of nodes drawing substantial power even in idle or standby modes. Although powering down unused nodes can improve energy efficiency, choosing the wrong time to do so can degrade quality of service by delaying job execution. Machine learning, in particular reinforcement learning (RL), has shown promise in determining optimal times to switch nodes on or off. In this study, we enhance the performance of a deep reinforcement learning (DRL) agent for HPC power management by integrating curriculum learning (CL), a training approach that introduces tasks with gradually increasing difficulty. Using the Batsim-py simulation framework, we compare the proposed CL-based agent to both a baseline DRL method (without CL) and the conventional fixed-time timeout strategy. Experimental results confirm that an easy-to-hard curriculum outperforms other training orders in terms of reducing wasted energy usage. The best agent achieves a 3.73% energy reduction over the baseline DRL method and a 4.66% improvement compared to the best timeout configuration (shutdown every 15 minutes of idle time). In addition, it reduces average job waiting time by 9.24% and maintains a higher job-filling rate, indicating more effective resource utilization. Sensitivity tests across various switch-on durations, power levels, and cluster sizes further reveal the agent's adaptability to changing system parameters without retraining. These findings demonstrate that curriculum learning can significantly improve DRL-based power management in HPC, balancing energy savings, quality of service, and robustness to diverse configurations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。