LLM能从自然语言中推导出高性能计算调度最优解,但多数仍难精准计算时序与依赖。
Evaluating Large Language Models for Workload Mapping and Scheduling in Heterogeneous HPC Systems
- 用自然语言描述系统与任务约束,让LLM直接规划调度方案。
- 3个模型达成最优9小时20秒,12个在2分钟内接近最优。
- 适合用作可解释的决策辅助,而非完全自主的优化引擎。
大型语言模型(LLMs)日益被探索用于推理能力,但其从自然语言中执行结构化、约束型优化的能力仍不明确。本研究评估了21个公开可用的LLM在典型异构高性能计算(HPC)工作负载映射与调度问题上的表现。每个模型接收相同的系统节点、任务需求及调度约束的文本描述,需将任务分配至节点、计算总完工时间,并解释推理过程。以人工推导的9小时20秒解析最优解为基准。3个模型精确复现该最优解,12个结果在基准值2分钟内,6个产生次优方案且存在算术或依赖性错误。所有模型均生成可行的任务-节点映射,但仅约一半严格遵守约束。19个模型生成部分可执行验证代码,18个提供连贯分步推理,即使存在逻辑错误也展现出强可解释性。整体表明当前LLM在组合优化中的推理边界:顶尖模型可直接从自然语言重建最优调度,但多数仍难以精确处理时间计算、数据传输算术及依赖关系强制。结果凸显LLM作为可解释协作者在优化与决策支持中的潜力,而非自主求解器。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly explored for their reasoning capabilities, yet their ability to perform structured, constraint-based optimization from natural language remains insufficiently understood. This study evaluates twenty-one publicly available LLMs on a representative heterogeneous high-performance computing (HPC) workload mapping and scheduling problem. Each model received the same textual description of system nodes, task requirements, and scheduling constraints, and was required to assign tasks to nodes, compute the total makespan, and explain its reasoning. A manually derived analytical optimum of nine hours and twenty seconds served as the ground truth reference. Three models exactly reproduced the analytical optimum while satisfying all constraints, twelve achieved near-optimal results within two minutes of the reference, and six produced suboptimal schedules with arithmetic or dependency errors. All models generated feasible task-to-node mappings, though only about half maintained strict constraint adherence. Nineteen models produced partially executable verification code, and eighteen provided coherent step-by-step reasoning, demonstrating strong interpretability even when logical errors occurred. Overall, the results define the current capability boundary of LLM reasoning in combinatorial optimization: leading models can reconstruct optimal schedules directly from natural language, but most still struggle with precise timing, data transfer arithmetic, and dependency enforcement. These findings highlight the potential of LLMs as explainable co-pilots for optimization and decision-support tasks rather than autonomous solvers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。