改进贝叶斯优化的迁移学习方法，提升高效寻优能力。
An Empirical Study on Ensemble-Based Transfer Learning Bayesian Optimisation with Mixed Variable Types
- 用正则回归加权重约束构建集成代理模型预测。
- 热启动初始化与正权重约束显著提升优化性能。
- 提出三个新基准，适合优化算法研究者参考。
贝叶斯优化是一种高效的全局最优化方法，适用于昂贵的黑箱目标函数。通过将相关问题的历史数据用于迁移学习，可改善贝叶斯优化的性能。本文对多种基于集成的迁移学习贝叶斯优化方法及其流程组件进行了实证分析。我们拓展了现有工作，提出了若干具体流程组件，并构建了三个新的实时迁移学习贝叶斯优化基准。特别地，我们提出一种基于正则回归且权重受限为正的加权策略来处理集成代理模型的预测，以及一个在迁移学习无益时的应对机制。实验发现，热启动初始化和限制集成代理模型权重为正值是提升迁移学习贝叶斯优化性能的两个关键组件。
原文摘要 · Abstract (English)
Bayesian optimisation is a sample efficient method for finding a global optimum of expensive black-box objective functions. Historic datasets from related problems can be exploited to help improve performance of Bayesian optimisation by adapting transfer learning methods to various components of the Bayesian optimisation pipeline. In this study we perform an empirical analysis of various ensemble-based transfer learning Bayesian optimisation methods and pipeline components. We expand on previous work in the literature by contributing some specific pipeline components, and three new real-time transfer learning Bayesian optimisation benchmarks. In particular we propose to use a weighting strategy for ensemble surrogate model predictions based on regularised regression with weights constrained to be positive, and a related component for handling the case when transfer learning is not improving Bayesian optimisation performance. We find that in general, two components that help improve transfer learning Bayesian optimisation performance are warm start initialisation and constraining weights used with ensemble surrogate model to be positive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。