让大模型专注设计算法，参数调优交给专门工具，省时省钱。
In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics
- 大模型负责生成新算法结构，参数调优由独立优化流程完成。
- 在多个基准问题上性能优于或相当现有方法，计算成本大幅降低。
- 适合需要高效生成优化算法的研究者与工程实践者。
大语言模型（LLMs）在自动生成和优化（元）启发式算法方面展现出巨大潜力，但在调优生成算法的超参数时通常效率低下，常需大量查询，导致高计算与资金成本。本文提出一种新型混合方法 LLaMEA-HPO，将开源的 LLaMEA（大语言模型进化算法）框架与闭环超参数优化（HPO）流程结合。通过将超参数调优任务交由 HPO 程序执行，LLaMEA-HPO 使大模型可专注于生成新颖的算法结构，显著减少所需的大模型查询次数，提升整体优化效率。我们在在线装箱、黑盒优化和旅行商问题等基准任务上进行实证验证，结果表明：该框架在性能上优于或相当于现有基于 LLM 的方法，同时显著降低计算开销。本工作强调了在基于 LLM 的代码优化中，将算法创新与结构搜索与参数调优分离的重要性，并提供了一种可扩展的高效方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown great potential in automatically generating and optimizing (meta)heuristics, making them valuable tools in heuristic optimization tasks. However, LLMs are generally inefficient when it comes to fine-tuning hyper-parameters of the generated algorithms, often requiring excessive queries that lead to high computational and financial costs. This paper presents a novel hybrid approach, LLaMEA-HPO, which integrates the open source LLaMEA (Large Language Model Evolutionary Algorithm) framework with a Hyper-Parameter Optimization (HPO) procedure in the loop. By offloading hyper-parameter tuning to an HPO procedure, the LLaMEA-HPO framework allows the LLM to focus on generating novel algorithmic structures, reducing the number of required LLM queries and improving the overall efficiency of the optimization process. We empirically validate the proposed hybrid framework on benchmark problems, including Online Bin Packing, Black-Box Optimization, and the Traveling Salesperson Problem. Our results demonstrate that LLaMEA-HPO achieves superior or comparable performance compared to existing LLM-driven frameworks while significantly reducing computational costs. This work highlights the importance of separating algorithmic innovation and structural code search from parameter tuning in LLM-driven code optimization and offers a scalable approach to improve the efficiency and effectiveness of LLM-based code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。