用贝叶斯优化融合序列二次规划，提升黑箱优化效率。
BayeSQP: Bayesian Optimization through Sequential Quadratic Programming
- 用高斯过程联合建模目标函数与约束的值、梯度和海森矩阵。
- 每轮构建考虑不确定性的次优子问题，求解后沿方向采样新点。
- 适合高维黑箱优化，兼具理论严谨性与实际灵活性。
我们提出BayeSQP，一种融合序列二次规划与贝叶斯优化思想的新型黑箱优化算法。该方法利用二阶高斯过程代理模型，仅通过零阶信息联合建模目标函数与约束的函数值、梯度及海森矩阵。每轮迭代中，基于高斯过程后验估计构造局部子问题，并求解以获得搜索方向。关键在于，子问题显式纳入函数与导数估计的不确定性，从而在模型不确定性下以高概率实现改进，且可转化为可求解的二阶锥规划。随后通过受限Thompson采样进行一维线搜索，确定下一次评估点。实验表明，该算法在特定高维场景下优于当前最优方法。本算法提供了一个原则性强、灵活度高的框架，弥合了经典优化与现代黑箱优化之间的鸿沟。
原文摘要 · Abstract (English)
We introduce BayeSQP, a novel algorithm for general black-box optimization that merges the structure of sequential quadratic programming with concepts from Bayesian optimization. BayeSQP employs second-order Gaussian process surrogates for both the objective and constraints to jointly model the function values, gradients, and Hessian from only zero-order information. At each iteration, a local subproblem is constructed using the GP posterior estimates and solved to obtain a search direction. Crucially, the formulation of the subproblem explicitly incorporates uncertainty in both the function and derivative estimates, resulting in a tractable second-order cone program for high probability improvements under model uncertainty. A subsequent one-dimensional line search via constrained Thompson sampling selects the next evaluation point. Empirical results show thatBayeSQP outperforms state-of-the-art methods in specific high-dimensional settings. Our algorithm offers a principled and flexible framework that bridges classical optimization techniques with modern approaches to black-box optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。