arXiv:2510.17899cs.LGcs.AI2025-10被引 1

用大模型自动生成适配具体任务的调优算法,性能比人工设计提升超70%。

Automated Algorithm Design for Auto-Tuning Optimizers

  • 让大模型根据问题特征生成定制化调优算法,边试边优化。
  • 在6种硬件上测试,平均性能比现有方法高30.7%,最优提升达72.4%。
  • 适合需要自动调优的高性能计算场景,尤其适合复杂搜索空间。

自动性能调优对优化高性能应用至关重要,但庞大的非规则搜索空间使手动探索不可行。传统方法依赖进化、退火或基于代理的优化器,而高效且鲁棒地找到近优配置仍具挑战。本文提出新范式:使用大语言模型(LLMs)自动生成针对具体调优问题的优化算法。我们构建框架,通过问题描述和搜索空间特征提示LLM,合成、测试并迭代优化专用优化器。这些生成算法在四个真实世界调优应用上,于六种硬件平台进行评估,并与两种主流调优框架中的最先进方法对比。结果显示,在生成阶段引入应用和搜索空间特异性信息,分别带来30.7%和14.6%的平均性能提升。此外,生成的优化器可媲美甚至超越人类设计的算法,最佳生成算法相较当前最先进方法平均提升72.4%。

原文摘要 · Abstract (English)

Automatic performance tuning (auto-tuning) is essential for optimizing high-performance applications, where vast and irregular search spaces make manual exploration infeasible. While auto-tuners traditionally rely on classical approaches such as evolutionary, annealing, or surrogate-based optimizers, designing algorithms that efficiently find near-optimal configurations robustly across diverse tasks is challenging. We propose a new paradigm: using large language models (LLMs) to automatically generate optimization algorithms tailored to auto-tuning problems. We introduce a framework that prompts LLMs with problem descriptions and search space characteristics to synthesize, test, and iteratively refine specialized optimizers. These generated algorithms are evaluated on four real-world auto-tuning applications across six hardware platforms and compared against the state-of-the-art in two contemporary auto-tuning frameworks. The evaluation demonstrates that providing additional application- and search space-specific information in the generation stage results in an average performance improvement of 30.7% and 14.6%, respectively. In addition, our results show that LLM-generated optimizers can rival, and in various cases outperform, existing human-designed algorithms, with our best-performing generated optimization algorithms achieving an average 72.4% improvement over state-of-the-art optimizers for auto-tuning.

自动调优大模型优化算法性能提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。