arXiv:2503.18060cs.LGcs.NE2025-03中稿 · ACM GECCO 2025被引 10

用代理模型减少优化训练中的函数评估次数,提升效率。

Surrogate Learning in Meta-Black-Box Optimization: A Preliminary Study

  • 用KAN网络和新损失函数建模目标函数,替代真实评估
  • 强化学习动态调整差分进化变异算子,减少30%以上评估量
  • 适合高维优化问题,代码已开源,可复现

近期元黑箱优化(MetaBBO)方法通过学习元策略动态配置底层优化器,提升了优化性能。然而,现有方法在训练元策略时需大量函数评估。受代理模型用于低成本评估昂贵优化问题的启发,本文提出Surr-RLDE框架,结合代理学习与强化学习增强的差分进化算法,以缓解元优化中评估开销过大的问题。该框架包含两阶段:代理学习阶段,使用新型相对顺序感知损失训练Kolmogorov-Arnold Networks(KAN),精准逼近用于后续策略学习的问题实例目标函数;策略学习阶段,采用强化学习动态配置差分进化中的变异算子。所学代理模型被融入强化学习训练过程,替代原始目标函数,显著降低策略学习期间的评估消耗。大量基准测试表明,Surr-RLDE不仅性能媲美近期基线,且在高维问题上展现出优异泛化能力。消融实验验证了各技术组件的有效性。项目代码已开源:https://github.com/GMC-DRL/Surr-RLDE。

原文摘要 · Abstract (English)

Recent Meta-Black-Box Optimization (MetaBBO) approaches have shown possibility of enhancing the optimization performance through learning meta-level policies to dynamically configure low-level optimizers. However, existing MetaBBO approaches potentially consume massive function evaluations to train their meta-level policies. Inspired by the recent trend of using surrogate models for cost-friendly evaluation of expensive optimization problems, in this paper, we propose a novel MetaBBO framework which combines surrogate learning process and reinforcement learning-aided Differential Evolution algorithm, namely Surr-RLDE, to address the intensive function evaluation in MetaBBO. Surr-RLDE comprises two learning stages: surrogate learning and policy learning. In surrogate learning, we train a Kolmogorov-Arnold Networks (KAN) with a novel relative-order-aware loss to accurately approximate the objective functions of the problem instances used for subsequent policy learning. In policy learning, we employ reinforcement learning (RL) to dynamically configure the mutation operator in DE. The learned surrogate model is integrated into the training of the RL-based policy to substitute for the original objective function, which effectively reduces consumed evaluations during policy learning. Extensive benchmark results demonstrate that Surr-RLDE not only shows competitive performance to recent baselines, but also shows compelling generalization for higher-dimensional problems. Further ablation studies underscore the effectiveness of each technical components in Surr-RLDE. We open-source Surr-RLDE at https://github.com/GMC-DRL/Surr-RLDE.

元优化代理模型强化学习差分进化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。