用牛顿法构建局部二次模型，提升高维贝叶斯优化采样效率。
Enhancing Trust-Region Bayesian Optimization via Newton Methods
- 用全局GP的梯度和海森矩阵构建局部二次模型，替代局部GP
- 在合成函数和真实任务上优于多种高维贝叶斯优化方法
- 适合高维、昂贵黑箱函数优化，尤其有梯度信息的场景
贝叶斯优化（BO）广泛用于高效优化昂贵的黑箱函数，但其在高维空间中的扩展仍具挑战。现有方法通过在多个局部信任域内进行标准贝叶斯优化（TuRBO），实现目标函数的异质建模并避免过度探索。然而，使用局部高斯过程（GPs）会降低采样效率。为在保持异质建模的同时提升采样效率，我们提出利用全局GP的梯度与海森矩阵构建多个局部二次模型，并通过求解带边界约束的二次规划选择新样本点。此外，我们解决了高维空间中GP梯度消失的问题。我们提供了收敛性分析，并通过实验表明，该方法显著提升了TuRBO的性能，在合成函数及真实应用中超越了多种高维贝叶斯优化技术。
原文摘要 · Abstract (English)
Bayesian Optimization (BO) has been widely applied to optimize expensive black-box functions while retaining sample efficiency. However, scaling BO to high-dimensional spaces remains challenging. Existing literature proposes performing standard BO in multiple local trust regions (TuRBO) for heterogeneous modeling of the objective function and avoiding over-exploration. Despite its advantages, using local Gaussian Processes (GPs) reduces sampling efficiency compared to a global GP. To enhance sampling efficiency while preserving heterogeneous modeling, we propose to construct multiple local quadratic models using gradients and Hessians from a global GP, and select new sample points by solving the bound-constrained quadratic program. Additionally, we address the issue of vanishing gradients of GPs in high-dimensional spaces. We provide a convergence analysis and demonstrate through experimental results that our method enhances the efficacy of TuRBO and outperforms a wide range of high-dimensional BO techniques on synthetic functions and real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。