用梯度信息增强贝叶斯神经网络，让优化更快更准。
Towards Scalable Bayesian Optimization via Gradient-Informed Bayesian Neural Networks
- 在训练贝叶斯神经网络时加入梯度信息，提升模型精度。
- 在高维问题上，优化收敛速度显著加快。
- 适合需要高效全局优化的复杂系统设计场景。
贝叶斯优化（BO）是一种广泛使用的数据驱动优化方法，通常依赖目标函数的零阶观测构建概率代理模型，以指导探索与利用过程寻找全局最优解。尽管高斯过程（GPs）常被用作未知目标函数的代理模型，但近年来研究表明，贝叶斯神经网络（BNNs）作为可扩展且灵活的替代方案具有潜力。此外，当可获得梯度信息时，将其融入GPs已被证明能提升BO性能。然而，将梯度信息用于BNN代理模型仍鲜有研究。本文通过自动微分技术，将梯度信息无缝集成到BNN训练中，从而构建更具信息量的代理模型。为此，我们提出一种梯度感知的损失函数，有效将局部梯度信息融入函数观测。该方法在多个经典基准测试中验证了其有效性，表现为更好的BNN预测性能以及随决策变量数量增加而更快的BO收敛速度。
原文摘要 · Abstract (English)
Bayesian optimization (BO) is a widely used method for data-driven optimization that generally relies on zeroth-order data of objective function to construct probabilistic surrogate models. These surrogates guide the exploration-exploitation process toward finding global optimum. While Gaussian processes (GPs) are commonly employed as surrogates of the unknown objective function, recent studies have highlighted the potential of Bayesian neural networks (BNNs) as scalable and flexible alternatives. Moreover, incorporating gradient observations into GPs, when available, has been shown to improve BO performance. However, the use of gradients within BNN surrogates remains unexplored. By leveraging automatic differentiation, gradient information can be seamlessly integrated into BNN training, resulting in more informative surrogates for BO. We propose a gradient-informed loss function for BNN training, effectively augmenting function observations with local gradient information. The effectiveness of this approach is demonstrated on well-known benchmarks in terms of improved BNN predictions and faster BO convergence as the number of decision variables increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。