arXiv:2505.12471stat.MLcs.LG2025-05被引 2

用沃瑟斯坦均值融合多个高斯过程,提升贝叶斯优化稳定性。

Wasserstein Barycenter Gaussian Process based Bayesian Optimization

  • 通过预设超参数组合构建多个高斯过程,取其沃瑟斯坦均值作为最终模型。
  • 在困难测试问题上收敛性显著优于传统方法,最优解搜索更可靠。
  • 适合追求高鲁棒性的黑箱优化场景,尤其对初始设置敏感的工程应用。

基于高斯过程的贝叶斯优化是处理不确定环境下的高效优化算法,以样本效率著称。然而,近年来研究发现其核心的高斯过程拟合过程存在关键缺陷:通常采用最大似然估计调整核函数超参数,但该方法在贝叶斯优化中表现不稳定,导致理论分析困难。受高斯过程与高斯分布的类比启发,本文提出一种新方法:预先设定一组超参数,拟合多个高斯过程,并将其合并为一个统一模型——即高斯过程的沃瑟斯坦均值。我们在“简单”和“棘手”的测试问题上验证该方法,结果表明,所提出的沃瑟斯坦均值高斯过程贝叶斯优化(WBGP-BO)在多数情况下能成功收敛至最优解,而传统方法在复杂问题上失败。

原文摘要 · Abstract (English)

Gaussian Process based Bayesian Optimization is a widely applied algorithm to learn and optimize under uncertainty, well-known for its sample efficiency. However, recently -- and more frequently -- research studies have empirically demonstrated that the Gaussian Process fitting procedure at its core could be its most relevant weakness. Fitting a Gaussian Process means tuning its kernel's hyperparameters to a set of observations, but the common Maximum Likelihood Estimation technique, usually appropriate for learning tasks, has shown different criticalities in Bayesian Optimization, making theoretical analysis of this algorithm an open challenge. Exploiting the analogy between Gaussian Processes and Gaussian Distributions, we present a new approach which uses a prefixed set of hyperparameters values to fit as many Gaussian Processes and then combines them into a unique model as a Wasserstein Barycenter of Gaussian Processes. We considered both "easy" test problems and others known to undermine the \textit{vanilla} Bayesian Optimization algorithm. The new method, namely Wasserstein Barycenter Gausssian Process based Bayesian Optimization (WBGP-BO), resulted promising and able to converge to the optimum, contrary to vanilla Bayesian Optimization, also on the most "tricky" test problems.

贝叶斯优化高斯过程优化算法概率建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。