用强化学习持续优化语言模型，加速发现高效算法。
Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning
- 让语言模型在进化搜索中不断学习改进自身生成策略。
- 在组合优化任务中显著加快找到更优算法的速度。
- 适合算法设计、AI for Science 领域的研究者参考。
在数学与计算机科学中，发现解决复杂问题的高效算法始终是重大挑战，需长期依赖人类专家经验。近年来，基于大语言模型（LLM）的进化搜索在多个领域（尤其是数学与优化）展现出加速算法发现的潜力。然而，现有方法将LLM视为静态生成器，未利用进化探索中获得的反馈信号来更新模型。本文提出将强化学习（RL）微调引入基于LLM的进化搜索，通过持续优化搜索算子（即LLM）来提升性能。实验表明，在组合优化任务中，结合RL与进化搜索可显著加速发现更优算法，验证了强化学习增强型进化策略在算法设计中的可行性与潜力。
原文摘要 · Abstract (English)
Discovering efficient algorithms for solving complex problems has been an outstanding challenge in mathematics and computer science, requiring substantial human expertise over the years. Recent advancements in evolutionary search with large language models (LLMs) have shown promise in accelerating the discovery of algorithms across various domains, particularly in mathematics and optimization. However, existing approaches treat the LLM as a static generator, missing the opportunity to update the model with the signal obtained from evolutionary exploration. In this work, we propose to augment LLM-based evolutionary search by continuously refining the search operator - the LLM - through reinforcement learning (RL) fine-tuning. Our method leverages evolutionary search as an exploration strategy to discover improved algorithms, while RL optimizes the LLM policy based on these discoveries. Our experiments on combinatorial optimization tasks demonstrate that integrating RL with evolutionary search accelerates the discovery of superior algorithms, showcasing the potential of RL-enhanced evolutionary strategies for algorithm design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。