用强化学习自动学特征,让差分进化算法自适应优化更高效
Reinforcement Learning-based Self-adaptive Differential Evolution through Automated Landscape Feature Learning
- 用注意力网络自动提取优化过程中的景观特征
- 在合成与真实场景中均优于多个先进算法和基线方法
- 适合需要自动调参的复杂优化任务研究者
近期,元黑箱优化(MetaBBO)方法通过元学习构建灵活通用的元策略,在低层优化中实现动态算法配置(DAC),显著提升传统黑箱优化器性能,并降低对领域专家的需求。然而,现有方法严重依赖人工设计特征提取,限制了学习效果。本文提出新型MetaBBO方法RLDE-AFL,支持元学习过程中自动特征学习,将可学习特征提取模块融入基于强化学习的差分进化(DE)框架,同步学习特征编码与元策略。具体地,设计基于尾数-指数嵌入的注意力神经网络,将低层优化中的解群体及其目标值转化为高表达力的景观特征。同时,引入包含多种DE算子的综合算法配置空间,结合强化学习辅助的DAC范式,释放行为多样性与性能潜力。大量基准测试表明,联合训练特征学习模块与DAC策略使RLDE-AFL在合成与真实BBO场景下均优于多个先进DE方法及最新MetaBBO基线。代码已开源:https://github.com/GMC-DRL/RLDE-AFL。
原文摘要 · Abstract (English)
Recently, Meta-Black-Box-Optimization (MetaBBO) methods significantly enhance the performance of traditional black-box optimizers through meta-learning flexible and generalizable meta-level policies that excel in dynamic algorithm configuration (DAC) tasks within the low-level optimization, reducing the expertise required to adapt optimizers for novel optimization tasks. Though promising, existing MetaBBO methods heavily rely on human-crafted feature extraction approach to secure learning effectiveness. To address this issue, this paper introduces a novel MetaBBO method that supports automated feature learning during the meta-learning process, termed as RLDE-AFL, which integrates a learnable feature extraction module into a reinforcement learning-based DE method to learn both the feature encoding and meta-level policy. Specifically, we design an attention-based neural network with mantissa-exponent based embedding to transform the solution populations and corresponding objective values during the low-level optimization into expressive landscape features. We further incorporate a comprehensive algorithm configuration space including diverse DE operators into a reinforcement learning-aided DAC paradigm to unleash the behavior diversity and performance of the proposed RLDE-AFL. Extensive benchmark results show that co-training the proposed feature learning module and DAC policy contributes to the superior optimization performance of RLDE-AFL to several advanced DE methods and recent MetaBBO baselines over both synthetic and realistic BBO scenarios. The source codes of RLDE-AFL are available at https://github.com/GMC-DRL/RLDE-AFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。