arXiv:2410.15625cs.LGcs.AI2024-10ICML被引 14

用AI自动优化并行程序调度,几分钟搞定专家几天的工作。

Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces

  • 通过专用语言和反馈机制,让AI理解系统运行细节并智能搜索最优调度方案。
  • 仅需10次迭代就超越传统方法1000次的性能,最快提升3.8倍。
  • 适合缺乏系统知识的科研人员快速提升计算效率,尤其在模拟仿真领域。

现代科学发现越来越依赖高性能计算进行复杂建模与仿真。提升并行程序性能的关键在于高效地将任务映射到处理器、数据映射到内存,这一过程由复杂的底层系统代码(即映射器)决定。开发高性能映射器需数天手动调优,对无系统背景的领域科学家构成重大障碍。我们提出一个框架,通过生成式优化自动化映射器开发,利用超越标量性能指标的丰富反馈。该方法包含代理-系统接口,其中领域特定语言(DSL)抽象底层系统复杂性并定义结构化搜索空间,以及AutoGuide机制,可将原始执行输出转化为可操作反馈。与仅依赖标量反馈的传统强化学习方法(如OpenTuner)不同,我们的方法在更少迭代中找到更优映射器。仅10次迭代即超越OpenTuner经1000次迭代的表现,实现最高3.8倍性能提升。在九个基准测试中,所生成映射器相比专家编写的映射器最高提速1.34倍,同时将调优时间从数天缩短至数分钟。

原文摘要 · Abstract (English)

Modern scientific discovery increasingly relies on high-performance computing for complex modeling and simulation. A key challenge in improving parallel program performance is efficiently mapping tasks to processors and data to memory, a process dictated by intricate, low-level system code known as mappers. Developing high-performance mappers demands days of manual tuning, posing a significant barrier for domain scientists without systems expertise. We introduce a framework that automates mapper development with generative optimization, leveraging richer feedback beyond scalar performance metrics. Our approach features the Agent-System Interface, which includes a Domain-Specific Language (DSL) to abstract away the low-level complexity of system code and define a structured search space, as well as AutoGuide, a mechanism that interprets raw execution output into actionable feedback. Unlike traditional reinforcement learning methods such as OpenTuner, which rely solely on scalar feedback, our method finds superior mappers in far fewer iterations. With just 10 iterations, it outperforms OpenTuner even after 1000 iterations, achieving 3.8X faster performance. Our approach finds mappers that surpass expert-written mappers by up to 1.34X speedup across nine benchmarks while reducing tuning time from days to minutes.

AI优化并行计算自动化调优

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。