高维黑箱优化新方法，用排序替代数值拟合，更快更准。
OPBO: Order-Preserving Bayesian Optimization
- 用神经网络保持目标函数的大小顺序，而非精确数值。
- 在500维以上问题中，性能远超传统贝叶斯优化方法。
- 适合高维、计算成本高的优化任务，如超参调优。
贝叶斯优化是求解昂贵黑箱优化问题的有效方法。现有方法多以高斯过程（GP）作为代理模型逼近目标函数，但其在高维空间（如维度超过500）中表现不佳，因依赖精确数值拟合，导致计算复杂度剧增。本文提出一种简单且有效的序保持贝叶斯优化（OPBO）方法，代理模型不再拟合函数值，而是保留目标函数的大小顺序。采用简单的序保持神经网络（OP NN）替代GP。此外，不从采集函数中搜索最优解，而是从有序集合中选取足够好的解，显著降低计算开销。实验表明，在高维（超过500维）黑箱优化问题上，所提OPBO显著优于基于回归神经网络和高斯过程的传统贝叶斯优化方法。代码已开源：https://github.com/pengwei222/OPBO。
原文摘要 · Abstract (English)
Bayesian optimization is an effective method for solving expensive black-box optimization problems. Most existing methods use Gaussian processes (GP) as the surrogate model for approximating the black-box objective function, it is well-known that it can fail in high-dimensional space (e.g., dimension over 500). We argue that the reliance of GP on precise numerical fitting is fundamentally ill-suited in high-dimensional space, where it leads to prohibitive computational complexity. In order to address this, we propose a simple order-preserving Bayesian optimization (OPBO) method, where the surrogate model preserves the order, instead of the value, of the black-box objective function. Then we can use a simple but effective OP neural network (NN) to replace GP as the surrogate model. Moreover, instead of searching for the best solution from the acquisition model, we select good-enough solutions in the ordinal set to reduce computational cost. The experimental results show that for high-dimensional (over 500) black-box optimization problems, the proposed OPBO significantly outperforms traditional BO methods based on regression NN and GP. The source code is available at https://github.com/pengwei222/OPBO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。