自动挑选最优核函数与采集函数组合,提升贝叶斯优化效率。
BOOST: A Data-Driven Framework for the Automated Joint Selection of Kernel and Acquisition Functions in Bayesian Optimization
- 基于数据驱动的离线评估,预测不同核与采集函数组合性能。
- 在合成任务和超参优化中显著优于固定参数方法。
- 适合需要高效调参且缺乏先验知识的研究者使用。
贝叶斯优化(BO)的性能高度依赖于核函数与采集函数等超参数的选择,但二者联合自动选择长期被忽视,导致实践者常依赖启发式或高成本手动调优。本文提出BOOST框架,通过离线评估阶段预测多种核-采集函数组合的性能,识别最有望的组合后再进行昂贵的评估。该方法将已观测点划分为参考集(建模用)与查询集(评估未见区域表现),类比机器学习中的训练-验证划分。在合成基准和机器学习超参数优化任务上的实验表明,BOOST持续优于固定超参数的BO,且与顶尖自适应方法相当,展现出对多样优化场景的鲁棒性。
原文摘要 · Abstract (English)
The performance of Bayesian optimization (BO), a highly sample-efficient method for expensive black-box problems, is critically governed by the selection of its hyperparameters, including the kernel and acquisition functions. This presents a significant practical challenge: an inappropriate combination of these can lead to poor performance and wasted evaluations. While individual improvements to kernel functions and acquisition functions have been actively explored, the joint and autonomous selection of the best pair of these fundamental hyperparameters has been largely overlooked. This forced practitioners to rely on heuristics or costly manual training. In this work, we propose a framework, BOOST (Bayesian Optimization with Optimal Kernel and Acquisition Function Selection Technique), that automates this selection. BOOST utilizes a simple offline evaluation stage to predict the performance of various kernel-acquisition function pairs and identify the most promising pair before committing to the expensive evaluation process. BOOST is a data-driven strategy selection procedure that evaluates kernel-acquisition pairs based on their empirical performance on the data-in-hand. At each iteration, previously observed points are partitioned into a reference set and a query set. These subsets play roles analogous to training and validation sets in machine learning: the reference set is used for model construction, while the query set represents unseen regions to retrospectively evaluate how effectively each candidate strategy progresses toward the target value. Experiments on synthetic benchmarks and machine learning hyperparameter optimization tasks demonstrate that BOOST consistently improves over fixed-hyperparameter BO and remains competitive with state-of-the-art adaptive methods, highlighting its robustness across diverse landscapes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。