用分布式强化学习优化超算多用户任务调度,提升效率与扩展性。
Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems
- 采用去中心化分布式近端策略优化算法,支持大规模并行训练。
- 在超过1150万条真实任务日志上验证,显著降低等待时间并提升资源利用率。
- 适合需要高扩展性调度系统的科研与工业级超算场景。
高性能计算(HPC)环境中的资源分配面临复杂挑战,调度算法需兼顾系统资源高效分配与多项性能指标,如任务等待时间和系统利用率。尽管当前主流仍为基于规则的调度算法,但随着系统异构性和规模持续增长,传统方法在最小化等待时间与最大化利用率方面面临效率与灵活性瓶颈。近期研究尝试利用强化学习(RL)构建更智能、自适应的调度策略,涵盖DQN、PPO等算法及图神经网络与RL结合的混合方法。然而,这些方法普遍依赖小规模数据集,且在使用大规模数据时存在可扩展性问题。本文提出一种基于新型去中心化分布式近端策略优化(DD-PPO)的调度算法,支持多工作节点大规模分布式训练,无需每步同步参数。通过消除对集中式策略更新的依赖,该方法显著提升可扩展性、训练效率与样本利用率。实验基于超过1150万条真实HPC任务日志进行验证,结果表明,相比传统规则调度器和现有基于RL的调度方法,DD-PPO在调度性能上均有显著提升。
原文摘要 · Abstract (English)
Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system utilization. While traditional rule-based scheduling algorithms dominate the current deployments of HPC systems, the increasing heterogeneity and scale of those systems is expected to challenge the efficiency and flexibility of those algorithms in minimizing job wait time and maximizing utilization. Recent research efforts have focused on leveraging advancements in Reinforcement Learning (RL) to develop more adaptable and intelligent scheduling strategies. Recent RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, and these methods face scalability issues when using large datasets. This study introduces a novel RL-based scheduler utilizing the Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. The validation dataset leveraged over 11.5 million real HPC job traces for comparing DD-PPO performance between traditional and advanced scheduling approaches, and the experimental results demonstrate improved scheduling performance in comparison to both rule-based schedulers and existing RL-based scheduling algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。