让大模型专攻结构设计，数值优化交给专门工具，提升效率。
A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization
- 大模型只负责设计程序骨架，留出空位给数值优化填补。
- 在三个科学任务中，混合方法比纯大模型或纯数值优化更优。
- 框架灵活可插拔，适配各类优化器，适用于科研与系统设计。
在基于大语言模型的进化算法中，大模型作为单一算子同时更新结构组件（如控制流）和连续参数，虽然擅长前者，但在后者上效率低下，会浪费大量令牌在试错循环中进行离散跳跃。为此，我们提出一种混合嵌套搜索框架：外层由大模型生成结构草图并留下数值空缺，内层数值优化器负责填充这些空缺。外层与内层求解器均可替换，可与任意基于文本的优化器、零阶优化器（如CMA-ES）、基于梯度的方法或MCMC采样器组合使用。我们在三个科学领域验证该框架：(i) 在闭式测试函数上的元优化器；(ii) 面向系统研究与社会困境的代码策略；(iii) 近似贝叶斯推断任务。在所有任务中，混合优化器均优于纯大模型驱动搜索和纯数值优化基线。代码见：https://github.com/vicgalle/hybrid-nested-search
原文摘要 · Abstract (English)
In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components (like control flow) and continuous parameters. While LLMs can be good at the first, they are not efficient at the second, wasting tokens taking discrete jumps inside a trial and error loop. We resolve this by formalizing a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an inner numerical optimizer tunes the sketch. Both the outer and inner solvers are pluggable: any text-based optimizer can be combined with a zero-order optimizer (CMA-ES), gradient-based routines, or MCMC samplers. We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas; and (iii) approximate Bayesian inference tasks. Across all three, the hybrid optimizer is superior to both vanilla LLM-driven search and pure numerical optimization baselines. Code at: https://github.com/vicgalle/hybrid-nested-search
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。