用高质量算法示例作先验，提升大模型自动设计算法的性能。
From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors
- 以优质算法示例作为先验知识引导大模型生成代码。
- 在pbo和bbob两个黑箱优化基准上性能显著提升。
- 适合关注大模型自动化算法设计的研究者阅读。
大型语言模型（LLMs）已广泛用于自动化算法设计，在多个领域展现出生成与演化算法的强大能力。现有研究多聚焦于解决特定问题的有效性，搜索策略主要依赖自适应提示设计。本文通过分析提示中每个词元对LLM生成算法代码的贡献，发现提供高质量算法示例可显著提升LLM驱动优化的表现。基于此洞察，我们提出利用基准算法作为先验指导LLM优化，并在两个黑箱优化基准——伪布尔优化套件（pbo）和黑箱优化套件（bbob）上验证了优越性能。结果表明，融合基准测试研究能有效提升LLM驱动黑箱优化方法的效率与鲁棒性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have already been widely adopted for automated algorithm design, demonstrating strong abilities in generating and evolving algorithms across various fields. Existing work has largely focused on examining their effectiveness in solving specific problems, with search strategies primarily guided by adaptive prompt designs. In this paper, through investigating the token-wise attribution of the prompts to LLM-generated algorithmic codes, we show that providing high-quality algorithmic code examples can substantially improve the performance of the LLM-driven optimization. Building upon this insight, we propose leveraging prior benchmark algorithms to guide LLM-driven optimization and demonstrate superior performance on two black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization suite (bbob). Our findings highlight the value of integrating benchmarking studies to enhance both efficiency and robustness of the LLM-driven black-box optimization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。