用在线推荐系统动态选硬件，提升分布式计算资源效率
BanditWare: A Contextual Bandit-based Framework for Hardware Prediction
- 基于上下文多臂赌博机算法实时推荐最优硬件
- 在三个工作流上显著降低延迟并提升资源利用率
- 无需历史数据，适合各类用户快速部署使用
分布式计算系统对满足现代应用需求至关重要，但由单机转向分布式环境面临诸多挑战。共享系统中资源分配不当会导致资源争用、系统不稳定、性能下降、优先级反转、利用率低、延迟增加及环境影响。我们提出BanditWare，一种基于上下文多臂赌博机的在线推荐系统，能动态为应用选择最合适的硬件。该系统在探索与利用间取得平衡，根据实际应用表现逐步优化推荐结果，同时持续探索潜在更优选项。不同于依赖大量历史数据的传统统计与机器学习方法，BanditWare支持在线学习与实时适应，新负载到来时即刻更新策略。我们在三个工作流应用上评估：Cycles（农业科学工作流）、BurnPro3D（火灾科学网络平台）和矩阵乘法应用。系统设计支持与国家数据平台（NDP）无缝集成，使各类经验水平用户均可高效优化资源配置。
原文摘要 · Abstract (English)
Distributed computing systems are essential for meeting the demands of modern applications, yet transitioning from single-system to distributed environments presents significant challenges. Misallocating resources in shared systems can lead to resource contention, system instability, degraded performance, priority inversion, inefficient utilization, increased latency, and environmental impact. We present BanditWare, an online recommendation system that dynamically selects the most suitable hardware for applications using a contextual multi-armed bandit algorithm. BanditWare balances exploration and exploitation, gradually refining its hardware recommendations based on observed application performance while continuing to explore potentially better options. Unlike traditional statistical and machine learning approaches that rely heavily on large historical datasets, BanditWare operates online, learning and adapting in real-time as new workloads arrive. We evaluated BanditWare on three workflow applications: Cycles (an agricultural science scientific workflow) BurnPro3D (a web-based platform for fire science) and a matrix multiplication application. Designed for seamless integration with the National Data Platform (NDP), BanditWare enables users of all experience levels to optimize resource allocation efficiently.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。