用机器学习预测任务时长，让超算调度更高效
Duration-Informed Workload Scheduler
- 引入机器学习模块预测任务执行时长
- 实测平均等待时间降低约11%
- 适合超算中心与高性能计算研究者
高性能计算系统由众多子系统协同运行，其中工作负载调度器对持续提交的任务及时执行具有关键影响。高质量调度依赖于准确预知任务执行时长——这对用户而言非易事，可通过机器学习解决。本文设计了一种集成时长预测模块的调度器，基于顶级超算系统的作业轨迹进行评估，结果显示所有任务的平均等待时间减少约11%。更短的等待时间既提升了用户体验，也提高了系统整体吞吐效率。
原文摘要 · Abstract (English)
High-performance computing systems are complex machines whose behaviour is governed by the correct functioning of its many subsystems. Among these, the workload scheduler has a crucial impact on the timely execution of the jobs continuously submitted to the computing resources. Making high-quality scheduling decisions is contingent on knowing the duration of submitted jobs before their execution--a non-trivial task for users that can be tackled with Machine Learning. In this work, we devise a workload scheduler enhanced with a duration prediction module built via Machine Learning. We evaluate its effectiveness and show its performance using workload traces from a Tier-0 supercomputer, demonstrating a decrease in mean waiting time across all jobs of around 11%. Lower waiting times are directly connected to better quality of service from the users' point of view and higher turnaround from the system's perspective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。