用逆向工程生成多样化优化问题,提升元黑箱优化的泛化能力
Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering
- 通过自编码器将问题特征映射到二维隐空间,均匀采样生成多样隐表示
- 利用遗传编程搜索公式,使生成问题与隐表示的L2距离最小化
- 生成的Diverse-BBO数据集显著提升元优化器在新场景下的泛化性能
为减少设计优化算法所需的人工经验,现有元黑箱优化(MetaBBO)研究利用元学习的泛化能力,在预定义训练问题集上训练神经网络算法设计策略,以自动适配未见问题实例。当前主流选择是知名基准套件CoCo-BBOB,但其问题实例多样性有限,易导致元优化器过拟合,影响泛化性能。本文提出一种名为LSRE的实例生成方法,可生成多样化训练问题集以学习更通用的策略。首先训练一个自编码器,将高维问题特征映射至二维隐空间;在此空间中进行均匀网格采样,获得具有足够多样性的隐表示;再通过遗传编程搜索函数公式,使其与这些隐表示的L2距离最小,从而逆向生成多样化问题集,命名为Diverse-BBO。在Diverse-BBO上训练多种MetaBBO模型,并在合成或真实场景下评估其泛化性能。大量实验表明,Diverse-BBO在元优化器泛化性方面优于现有训练集。消融研究进一步验证了LSRE设计的有效性,并揭示了实例多样性与元优化器泛化之间的深层关系。
原文摘要 · Abstract (English)
To relieve intensive human-expertise required to design optimization algorithms, recent Meta-Black-Box Optimization (MetaBBO) researches leverage generalization strength of meta-learning to train neural network-based algorithm design policies over a predefined training problem set, which automates the adaptability of the low-level optimizers on unseen problem instances. Currently, a common training problem set choice in existing MetaBBOs is well-known benchmark suites CoCo-BBOB. Although such choice facilitates the MetaBBO's development, problem instances in CoCo-BBOB are more or less limited in diversity, raising the risk of overfitting of MetaBBOs, which might further results in poor generalization. In this paper, we propose an instance generation approach, termed as \textbf{LSRE}, which could generate diverse training problem instances for MetaBBOs to learn more generalizable policies. LSRE first trains an autoencoder which maps high-dimensional problem features into a 2-dimensional latent space. Uniform-grid sampling in this latent space leads to hidden representations of problem instances with sufficient diversity. By leveraging a genetic-programming approach to search function formulas with minimal L2-distance to these hidden representations, LSRE reverse engineers a diversified problem set, termed as \textbf{Diverse-BBO}. We validate the effectiveness of LSRE by training various MetaBBOs on Diverse-BBO and observe their generalization performances on either synthetic or realistic scenarios. Extensive experimental results underscore the superiority of Diverse-BBO to existing training set choices in MetaBBOs. Further ablation studies not only demonstrate the effectiveness of design choices in LSRE, but also reveal interesting insights on instance diversity and MetaBBO's generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。