arXiv:2503.03625math.OCcs.LG2025-03被引 3

对比全局优化器在贝叶斯优化中的效果,发现盲目追求全局最优反而可能适得其反。

Deterministic Global Optimization of the Acquisition Function in Bayesian Optimization: To Do or Not To Do?

  • 用确定性全局求解器MAiNGO替代局部/随机方法优化采集函数
  • 在高探索性采集函数下,三者收敛概率相近但MAiNGO需更多迭代
  • 过度追求采集函数全局最优时易陷入局部解,不如随机方法稳健

贝叶斯优化(BO)依赖高斯过程对采集函数进行优化以决定采样点。本文比较了确定性全局求解器MAiNGO与传统局部求解器L-BFGS-B及多起点随机求解器在采集函数优化中的表现。为兼顾计算效率,设定时间上限后取最佳解作为结果。实验以Muller-Brown势函数为基准,采用下置信界(LCB)采集函数,并扩展至三个其他基准函数。统计分析显示:当采集函数更偏向利用(exploitative)时,使用MAiNGO的BO可在更少迭代内收敛;但当数据多样性不足或采集函数过于激进时,MAiNGO比局部求解器更易收敛到黑箱函数的局部近优解。而L-BFGS-B和多起点方法通过引入采样点选择的随机性,增强了对未探索区域的探索能力,降低对采集函数超参数的依赖。最终结论:对不佳采集函数的次优优化,可能优于对其的精确优化。在高度探索性采集函数下,三种方法达到全局近优解的概率相当,但MAiNGO通常需要更多迭代才能收敛。

原文摘要 · Abstract (English)

Bayesian Optimization (BO) with Gaussian Processes relies on optimizing an acquisition function to determine sampling. We investigate the advantages and disadvantages of using a deterministic global solver (MAiNGO) compared to conventional local and stochastic global solvers (L-BFGS-B and multi-start, respectively) for the optimization of the acquisition function. For CPU efficiency, we set a time limit for MAiNGO, taking the best point as optimal. We perform repeated numerical experiments, initially using the Muller-Brown potential as a benchmark function, utilizing the lower confidence bound acquisition function; we further validate our findings with three alternative benchmark functions. Statistical analysis reveals that when the acquisition function is more exploitative (as opposed to exploratory), BO with MAiNGO converges in fewer iterations than with the local solvers. However, when the dataset lacks diversity, or when the acquisition function is overly exploitative, BO with MAiNGO, compared to the local solvers, is more likely to converge to a local rather than a global ly near-optimal solution of the black-box function. L-BFGS-B and multi-start mitigate this risk in BO by introducing stochasticity in the selection of the next sampling point, which enhances the exploration of uncharted regions in the search space and reduces dependence on acquisition function hyperparameters. Ultimately, suboptimal optimization of poorly chosen acquisition functions may be preferable to their optimal solution. When the acquisition function is more exploratory, BO with MAiNGO, multi-start, and L-BFGS-B achieve comparable probabilities of convergence to a globally near-optimal solution (although BO with MAiNGO may require more iterations to converge under these conditions).

贝叶斯优化全局优化采集函数随机性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。