让搜索模型在测试时自适应优化策略，提升进化搜索效率。
PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
- 用可训练的顾问模型分离策略与执行，动态生成和评估候选方案。
- 在进化早期用群体相对反馈学习大方向，后期聚焦最优解贡献以稳定优化。
- 适合需要高效试错的工程与科研场景，尤其在昂贵评估任务中表现突出。
大型语言模型已成为进化搜索的核心驱动力，但多数系统依赖固定提示生成的策略来采样新候选，难以适应实际工程与研究中评价成本高、需学习特定任务搜索动态的挑战。本文提出 PACEvolve++，一种基于顾问模型的强化学习框架，用于进化搜索代理在测试时的策略自适应。该框架将战略决策与实现解耦：可训练的顾问模型负责生成、评估并选择假设，而更强的前沿模型则将选定假设转化为可执行候选。为应对非平稳反馈，提出分阶段自适应方法：进化初期使用群体相对反馈学习广泛搜索偏好；后期随着奖励差距压缩，转而强调最佳 k 选一的前沿贡献，以支持稳定精炼。在专家并行负载均衡、序列推荐和蛋白质适应度外推任务中，PACEvolve++ 显著优于现有前沿模型框架，实现更快收敛，并稳定了进化搜索中的测试时训练。
原文摘要 · Abstract (English)
Large language models have become drivers of evolutionary search, but most systems rely on a fixed, prompt-elicited policy to sample next candidates. This limits adaptation in practical engineering and research tasks, where evaluations are expensive, and progress depends on learning task-specific search dynamics. We introduce PACEvolve++, an advisor-model reinforcement learning framework for test-time policy adaptation in evolutionary search agents. PACEvolve++ decouples strategic search decisions from implementation: a trainable advisor generates, assesses, and selects hypotheses, while a stronger frontier model translates selected hypotheses into executable candidates. To train the advisor under non-stationary feedback, we propose a phase-adaptive approach that adapts its optimization strategy to different phases of the evolutionary process. Early in evolution, it uses group-relative feedback to learn broad search preferences; later, as reward gaps compress, it emphasizes best-of-$k$ frontier contribution to support stable refinement. Across expert-parallel load balancing, sequential recommendation, and protein fitness extrapolation, PACEvolve++ outperforms the state-of-the-art evolutionary search framework with frontier models, achieving faster convergence and stabilizing test-time training during evolutionary search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。