用机器学习预测任务与节点的匹配关系，提升集群调度效率。
Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency
- 通过分析任务约束和节点属性，构建特征数据集用于模型训练。
- 采用集成投票模型，对单一适配节点的任务实现98%准确率。
- 适合关注大规模集群资源调度优化的研究者与工程师。
本研究探讨机器学习算法如何辅助工作负载分配策略，通过识别具有节点亲和性约束的任务（称为约束算子），这些算子将任务执行限制在少数节点上。基于真实世界谷歌集群数据（GCD）的工作负载轨迹和AGOCS框架，研究提取节点属性与任务约束，并分析其以确定合适的节点-任务匹配。重点关注可在单个节点或少于12.5k节点中的不到一千个节点上执行的任务。任务约束算子被压缩并经过独热编码预处理，作为训练数据集的特征。评估了多种机器学习分类器，包括人工神经网络、K近邻、决策树、朴素贝叶斯、岭回归、自适应提升和袋装法。最终的集成投票分类器模型在单一合适节点的任务上达到98%准确率，误分类率为1.5%-1.8%。
原文摘要 · Abstract (English)
This research investigates how Machine Learning (ML) algorithms can assist in workload allocation strategies by detecting tasks with node affinity operators (referred to as constraint operators), which constrain their execution to a limited number of nodes. Using real-world Google Cluster Data (GCD) workload traces and the AGOCS framework, the study extracts node attributes and task constraints, then analyses them to identify suitable node-task pairings. It focuses on tasks that can be executed on either a single node or fewer than a thousand out of 12.5k nodes in the analysed GCD cluster. Task constraint operators are compacted, pre-processed with one-hot encoding, and used as features in a training dataset. Various ML classifiers, including Artificial Neural Networks, K-Nearest Neighbours, Decision Trees, Naive Bayes, Ridge Regression, Adaptive Boosting, and Bagging, are fine-tuned and assessed for accuracy and F1-scores. The final ensemble voting classifier model achieved 98% accuracy and a 1.5-1.8% misclassification rate for tasks with a single suitable node.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。