让大模型学会跨任务复用编程经验,提升算法发现效率。
$\varepsilon$-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

- 用自然语言摘要存储成功策略,实现不同任务间知识迁移。
- 在8个任务上平均提升8.7%搜索效果,早期收敛快9.4%。
- 自适应门控防止错误记忆干扰,适合复杂优化场景研究者。
基于大模型的程序演化系统(如 FunSearch、AlphaEvolve)在发现新算法方面表现优异,但通常独立优化每个任务,完成后丢弃搜索经验。本文提出 $ε$-MemEvo 框架,实现大模型程序演化中的跨任务知识迁移。该框架将过往经验以任务无关的策略记忆形式存储:即对成功算法策略的紧凑自然语言摘要,而非原始代码,从而支持跨不同 API 与评估器的任务迁移。为避免语义不匹配导致的负迁移,$ε$-MemEvo 引入自适应注入门控,动态决定是否注入记忆及其强度。我们在8个涵盖数学优化与系统工程的多样化基准上进行评估,采用内容级留一法(Leave-One-Out),排除目标任务的记忆条目。在主干 GPT-5 上,$ε$-MemEvo 在全部8个任务上均优于 AdaEvolve,平均相对增益达+8.7%,早期收敛速度平均提升+9.4%。消融实验表明,盲目注入记忆可能引发灾难性失败,而自适应门控在所有五个消融任务中始终安全有效。更新后的后验分布具有可解释性:在搜索改善阶段偏好跳过,在早晚期平台期则从跳过转向提示。这些性能提升仅带来低于1%的计算开销。
原文摘要 · Abstract (English)
LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion. We introduce $\varepsilon$-MemEvo, a framework for cross-task knowledge transfer in LLM program evolution. $\varepsilon$-MemEvo stores prior experience as task-agnostic tactic memories: compact natural-language summaries of successful algorithmic strategies rather than raw code, enabling transfer across tasks with different APIs and evaluators. To avoid negative transfer from semantically mismatched memories, $\varepsilon$-MemEvo uses an adaptive injection gate that decides whether retrieved memories should be injected, and at what intensity. We evaluate $\varepsilon$-MemEvo on 8 diverse optimization benchmarks spanning mathematical optimization and systems engineering, using a content-level Leave-One-Out protocol that excludes target-task memory entries. On the primary GPT-5 backbone, $\varepsilon$-MemEvo improves AUCC over AdaEvolve on all 8 tasks, with a mean relative gain of +8.7%, and improves early-stage convergence by +9.4% on average. Ablations show that naive memory injection can fail catastrophically, while adaptive gating remains safe across all five ablation tasks. The data-updated posterior is interpretable in observed states: it favors skip during improving search and shifts from skip to hint across early and late plateaus. These gains incur less than 1% computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。