用分段线性近似实现高斯过程采集函数的全局优化
Global Optimization of Gaussian Process Acquisition Functions Using a Piecewise-Linear Kernel Approximation
- 将核函数分段线性化，转化为混合整数二次规划求解
- 在合成函数和超参调优任务中实现比采样法更优的收敛性能
- 适合需要严格全局最优的工业级优化场景
贝叶斯优化依赖迭代构建并优化采集函数，但该优化本身是难解的非凸问题。现有算法多采用采样或梯度方法，无法保证收敛至全局最优。本文提出基于混合整数规划（MIP）的全局优化框架，通过引入分段线性核近似（PK-MIQP），将任意平稳或点积核的不确定性采集函数转化为对应的混合整数二次规划形式。理论分析了该近似下的后悔界，并在合成函数、约束基准测试及超参数调优任务上验证了其有效性，显著提升优化精度。
原文摘要 · Abstract (English)
Bayesian optimization relies on iteratively constructing and optimizing an acquisition function. The latter turns out to be a challenging, non-convex optimization problem itself. Despite the relative importance of this step, most algorithms employ sampling- or gradient-based methods, which do not provably converge to global optima. This work investigates mixed-integer programming (MIP) as a paradigm for global acquisition function optimization. Specifically, our Piecewise-linear Kernel Mixed Integer Quadratic Programming (PK-MIQP) formulation introduces a piecewise-linear approximation for Gaussian process kernels and admits a corresponding MIQP representation for acquisition functions. The proposed method is applicable to uncertainty-based acquisition functions for any stationary or dot-product kernel. We analyze the theoretical regret bounds of the proposed approximation, and empirically demonstrate the framework on synthetic functions, constrained benchmarks, and a hyperparameter tuning task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。