用可学习框架融合历史与实时优化知识,让进化算法自动调参更高效。
Learning Evolution via Optimization Knowledge Adaptation
- 通过注意力机制参数化进化算子,实现知识驱动的自适应更新
- 在12个迁移场景中优于主流方法,无先验时也超越现有可学习进化算法
- 适合需要动态优化的自动化机器学习、模型调参场景
进化算法的迭代搜索过程蕴含了历史种群与适应度评估中的优化知识。有效利用这些知识对实现知识迁移与在线自适应至关重要。然而,现有研究通常孤立处理这两项目标,存在局限:进化序列迁移优化常无法充分利用先验知识,而自适应策略仅能调整特定进化算子。为此,我们提出优化知识自适应进化模型(OKAEM),一个统一的可学习进化框架,能基于可用优化知识自适应更新参数。通过注意力机制参数化进化算子,OKAEM实现可学习的更新规则,分两阶段利用知识:预训练阶段整合大量先验知识以实现高效迁移,自适应优化阶段则基于实时知识动态调整参数。实验表明,OKAEM在12个迁移场景中显著优于现有先进序列迁移方法,且在无先验条件下,仅凭自调优机制即超越先进可学习进化算法。此外,在视觉-语言模型提示调优中展现实际价值,消融实验验证可学习组件必要性,可视化分析揭示模型能自主发现可解释的进化规律。代码已公开于 https://gitee.com/Anonymity_Paper/code-of-okaem。
原文摘要 · Abstract (English)
The iterative search process of evolutionary algorithms (EAs) encapsulates optimization knowledge within historical populations and fitness evaluations. Effective utilization of this knowledge is crucial for facilitating knowledge transfer and online adaptation. However, current research typically addresses these goals in isolation and faces distinct limitations: evolutionary sequential transfer optimization often suffers from incomplete utilization of prior knowledge, while adaptive strategies, utilizing real-time knowledge, are limited to tailoring specific evolutionary operators. To simultaneously achieve these two capabilities, we introduce the Optimization Knowledge Adaptation Evolutionary Model (OKAEM), a unified learnable evolutionary framework capable of adaptively updating parameters based on available optimization knowledge. By parameterizing evolutionary operators via attention mechanisms, OKAEM enables learnable update rules that facilitate the utilization of optimization knowledge via two phases: pre-training to integrate extensive prior knowledge for efficient transfer, and adaptive optimization to dynamically update parameters based on real-time knowledge. Experimental results confirm that OKAEM significantly outperforms state-of-the-art sequential transfer methods across 12 transfer scenarios via pre-training, and surpasses advanced learnable EAs solely through its self-tuning mechanism in prior-free settings. Beyond demonstrating practical utility in prompt tuning for vision-language models, ablation studies validate the necessity of the learnable components, while visualization analyses reveal the model's capacity to autonomously discover interpretable evolutionary principles. The code can be accessed at https://gitee.com/Anonymity_Paper/code-of-okaem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。