用神经网络替代高斯过程,实现高效黑箱优化
Scalable neural network-based blackbox optimization
- 分离探索与利用策略,不依赖复杂的不确定性估计
- 在10~102维问题上,减少40%-60%评估次数
- 适合高维、大规模函数优化场景,运行速度提升10倍以上
贝叶斯优化(BO)广泛用于黑箱优化,依赖高斯过程(GP)模型和采集函数指导采样。但在高维空间或大量评估时,因GP计算复杂而面临可扩展性挑战。神经网络(NN)具备更好可扩展性且能建模复杂函数,催生了基于NN的BO方法。然而,这些方法通常依赖于对神经网络预测不确定性的估计,这在高维下计算成本高且复杂。为此,本文提出一种新方法——可扩展神经网络黑箱优化(SNBO),无需模型不确定性估计。SNBO通过分离探索与利用的采样标准,并自适应控制采样区域以提高效率。在10到102维的一系列优化问题上评估,相较于四种先进基线算法,SNBO在多数任务中取得更优函数值,同时减少40%-60%函数评估次数,运行时间降低至少一个数量级。
原文摘要 · Abstract (English)
Bayesian Optimization (BO) is a widely used approach for blackbox optimization that leverages a Gaussian process (GP) model and an acquisition function to guide future sampling. While effective in low-dimensional settings, BO faces scalability challenges in high-dimensional spaces and with large number of function evaluations due to the computational complexity of GP models. In contrast, neural networks (NNs) offer better scalability and can model complex functions, which led to the development of NN-based BO approaches. However, these methods typically rely on estimating model uncertainty in NN prediction -- a process that is often computationally intensive and complex, particularly in high dimensions. To address these limitations, a novel method, called scalable neural network-based blackbox optimization (SNBO), is proposed that does not rely on model uncertainty estimation. Specifically, SNBO adds new samples using separate criteria for exploration and exploitation, while adaptively controlling the sampling region to ensure efficient optimization. SNBO is evaluated on a range of optimization problems spanning from 10 to 102 dimensions and compared against four state-of-the-art baseline algorithms. Across the majority of test problems, SNBO attains function values better than the best-performing baseline algorithm, while requiring 40-60% fewer function evaluations and reducing the runtime by at least an order of magnitude.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。