让大模型自适应调整搜索策略,提升代码生成效率
AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
- 用累积改进信号动态调节搜索强度和资源分配
- 在185个开放问题上优于现有基线,显著减少计算浪费
- 适合需要高效探索的算法设计与系统优化场景
自动化程序生成正从一次生成转向推理时搜索,大型语言模型(LLMs)作为语义变异算子参与进化循环。然而,当前系统依赖静态调度,无法适应搜索过程的非平稳特性,导致资源浪费——停滞种群持续消耗算力,而有前景的探索方向却未被充分挖掘。我们提出AdaEvolve框架,将LLM驱动的进化重构为分层自适应优化问题。该框架通过‘累积改进信号’统一管理三个层面:局部自适应,动态调节种群内探索强度;全局自适应,基于带权赌博机调度跨种群资源分配;元引导,当进展停滞时,根据过往解及其改进情况生成新策略。实验表明,AdaEvolve在185个不同开放性优化问题(包括组合优化、系统优化与算法设计)中持续超越开源基线。
原文摘要 · Abstract (English)
The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operators within evolutionary loops. While effective, these systems are currently governed by static schedules that fail to account for the non-stationary dynamics of the search process. This rigidity results in substantial computational waste, as resources are indiscriminately allocated to stagnating populations while promising frontiers remain under-exploited. We introduce AdaEvolve, a framework that reformulates LLM-driven evolution as a hierarchical adaptive optimization problem. AdaEvolve uses an "accumulated improvement signal" to unify decisions across three levels: Local Adaptation, which dynamically modulates the exploration intensity within a population of solution candidates; Global Adaptation, which routes the global resource budget via bandit-based scheduling across different solution candidate populations; and Meta-Guidance which generates novel solution tactics based on the previously generated solutions and their corresponding improvements when the progress stalls. We demonstrate that AdaEvolve consistently outperforms the open-sourced baselines across 185 different open-ended optimization problems including combinatorial, systems optimization and algorithm design problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。