让大模型生成程序时共享通用结构,提升多任务发现效率与泛化能力。
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
- 分两阶段优化:先共建通用程序库,再针对具体任务微调。
- 在8类任务中,多数场景下性能优于单任务进化方法。
- 适合需要跨任务复用、数据少时防过拟合的程序生成场景。
近期基于大模型的演化搜索方法表明,通过迭代程序变异可发现强算法,但通常对每个任务独立优化,即使相关任务存在可复用结构。本文提出面向大模型引导程序发现的进化多任务优化(EMO),并设计EMO-STA(共享-再适应)框架:首先在任务族中演化共享的可执行程序档案,再将选中的共享候选适配到各目标任务。在EMO-STA中,探索了多种适配策略,包括从共享档案热启动、适配平均表现最优的共享程序,以及适配在每项任务上表现最佳的共享程序。在涵盖连续优化、几何构造、建模和算法优化的8个任务族中,EMO-STA在多数设置下优于同等计算量的单任务演化;其中,STA Best-Local提供最强的分布内适应,STA Best-Shared实现对未见任务的稳健迁移。计算资源分配实验表明,将大量家族级预算用于共享演化始终有益,且共享与适配预算大致平衡常为最优。除计算效率外,共享演化还能在低证据场景(如少量训练数据)中缓解过拟合,包括ARC任务和时间序列特征工程,因其更偏好跨任务泛化的程序,而非利用任务特异的脆弱特征。
原文摘要 · Abstract (English)
Recent LLM-guided evolutionary search methods have shown that iterative program mutation can discover strong algorithms, but they typically optimize each task independently, even when related tasks share reusable structure. We introduce Evolutionary Multi-Task Optimization (EMO) for LLM-guided program discovery, and propose EMO-STA (Shared-Then-Adapt), a two-stage framework that first evolves a shared archive of executable programs across a task family and then adapts selected shared candidates to each target task. Within EMO-STA, we explore multiple adaptation strategies, including warm-starting from the shared archive, adapting the best average shared program, and adapting the shared program that performs best on each target task. Across eight task families spanning continuous optimization, geometric construction, modeling, and algorithmic optimization, EMO-STA improves over matched-compute single-task evolution in most settings, with STA Best-Local providing the strongest in-distribution adaptation and STA Best-Shared yielding robust transfer to unseen tasks. Compute-allocation experiments show that allocating a substantial fraction of the family-level budget to shared evolution is consistently beneficial, with roughly balanced shared and adaptation budgets often being optimal. Beyond compute efficiency, we show that shared evolution can mitigate overfitting in low-evidence settings (e.g. few training data), including ARC tasks and time-series feature engineering, by favoring programs that generalize across all tasks rather than exploiting task-specific brittle artifacts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。