通过联合学习梯度与海森信息，实现高效高维贝叶斯优化。
NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Information
- 利用高斯过程同时建模梯度和海森矩阵，指导优化方向。
- 在上千变量问题上收敛更快，最终解优于现有方法。
- 适用于高维黑箱优化，尤其适合未知低维子空间场景。
贝叶斯优化对昂贵的黑箱问题有效，但在高维场景仍具挑战。本文提出NeST-BO，一种基于曲率感知的局部贝叶斯优化方法，通过联合学习梯度与海森信息的高斯过程代理模型，以一阶前瞻误差界选择评估点。该误差界随批次大小收缩，使步长误差趋于零；在良好行为邻域内可恢复类牛顿法的快速局部收敛性，同时借助标准保护机制保障全局收敛至驻点。为提升维度扩展性，将获取函数优化置于低维嵌入子空间（随机或学习所得），将主导的曲率学习成本从$O(d^2)$降至$O(m^2)$（$m \\< d$），同时保持步长目标精度。在高维合成及真实世界问题中，包括含数千变量且未知活跃子空间的情况，NeST-BO始终比最先进的局部与高维贝叶斯优化基线收敛更快、最终结果更优。
原文摘要 · Abstract (English)
Bayesian optimization (BO) is effective for expensive black-box problems but remains challenging in high dimensions. We propose NeST-BO, a curvature-aware local BO method that targets a (modified) Newton step by jointly learning gradient and Hessian information with Gaussian process (GP) surrogates, and selecting evaluations via a one-step lookahead bound on the Newton-step error. We show that this bound contracts with batch size, so NeST-BO drives the step error to zero; in well-behaved neighborhoods it recovers the fast local convergence behavior of inexact/modified Newton methods, while standard safeguards support global convergence to stationary points. To improve scaling with problem dimension, we optimize the acquisition in low-dimensional embedded subspaces (random or learned), reducing the dominant cost of learning curvature from $O(d^2)$ to $O(m^2)$ with $m \ll d$ while preserving step targeting. Across high-dimensional synthetic and real-world problems, including cases with thousands of variables and unknown active subspaces, NeST-BO consistently yields faster convergence and better final values than state-of-the-art local and high-dimensional BO baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。