用机器学习动态选OpenMP调度算法,提升高性能计算性能。
A Comparative Study of OpenMP Scheduling Algorithm Selection Strategies
- 结合专家经验与强化学习,动态选择最优调度算法。
- 在6个应用、3个系统上测试,强化学习表现优于固定策略。
- 适合需要自适应调度的科学计算与大数据程序开发者。
科学与数据科学应用日益复杂,对计算和内存的需求持续增长。现代高性能计算(HPC)系统在节点、设备和核心间具有高度并行性和异构性。为实现良好性能,有效的调度与负载均衡技术至关重要。OpenMP等并行编程框架提供了多种先进调度算法,以支持多样化的应用与平台,这引出了调度算法选择问题——即针对特定工作负载与系统特征,选择最合适的算法。本文探索了基于学习的OpenMP调度算法选择方法,提出了基于专家经验与强化学习(RL)的方法,并在六种应用和三种系统上进行了详尽的性能分析。结果表明,强化学习方法能够学习出高性能的调度决策,尽管需要大量探索,且奖励函数的设计起关键作用。相比之下,专家方法依赖先验知识,探索较少,但未必能为特定应用-系统组合找到最优算法。通过融合专家知识与强化学习,我们实现了性能提升与更强的适应性。总体而言,本工作证明,在执行期间动态选择调度算法对OpenMP应用既可行又有益。该方法还可扩展至基于MPI的程序,实现多级并行性下的调度优化。
原文摘要 · Abstract (English)
Scientific and data science applications are becoming increasingly complex, with growing computational and memory demands. Modern high performance computing (HPC) systems provide high parallelism and heterogeneity across nodes, devices, and cores. To achieve good performance, effective scheduling and load balancing techniques are essential. Parallel programming frameworks such as OpenMP now offer a variety of advanced scheduling algorithms to support diverse applications and platforms. This creates an instance of the scheduling algorithm selection problem, which involves identifying the most suitable algorithm for a given combination of workload and system characteristics. In this work, we explore learning-based approaches for selecting scheduling algorithms in OpenMP. We propose and evaluate expert-based and reinforcement learning (RL)-based methods, and conduct a detailed performance analysis across six applications and three systems. Our results show that RL methods are capable of learning high-performing scheduling decisions, although they require significant exploration, with the choice of reward function playing a key role. Expert-based methods, in contrast, rely on prior knowledge and involve less exploration, though they may not always identify the optimal algorithm for a specific application-system pair. By combining expert knowledge with RL-based learning, we achieve improved performance and greater adaptability. Overall, this work demonstrates that dynamic selection of scheduling algorithms during execution is both viable and beneficial for OpenMP applications. The approach can also be extended to MPI-based programs, enabling optimization of scheduling decisions across multiple levels of parallelism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。