解释高维贝叶斯优化为何有效，提出改进方法提升性能
Understanding High-Dimensional Bayesian Optimization
- 发现高维贝叶斯优化失败主因是核函数初始化导致梯度消失
- 最大似然估计长度尺度即可达顶尖性能，新方法MSR更优
- 适合解决真实世界高维优化问题的研究者参考
近期研究显示，简单贝叶斯优化（BO）方法在高维现实任务中表现良好，与以往结论相悖。本文探究其原因，识别出高维贝叶斯优化中的关键挑战。实证分析表明，高斯过程（GP）初始化引发的梯度消失是高维贝叶斯优化（HDBO）失败的主要原因，而促进局部搜索的方法更适用。研究发现，对GP长度尺度进行最大似然估计（MLE）即可实现顶尖性能。基于此，提出一种简单变体MSR，利用上述发现，在一系列真实应用中达到最优表现。通过针对性实验验证了核心结论。
原文摘要 · Abstract (English)
Recent work reported that simple Bayesian optimization (BO) methods perform well for high-dimensional real-world tasks, seemingly contradicting prior work and tribal knowledge. This paper investigates why. We identify underlying challenges that arise in high-dimensional BO and explain why recent methods succeed. Our empirical analysis shows that vanishing gradients caused by Gaussian process (GP) initialization schemes play a major role in the failures of high-dimensional Bayesian optimization (HDBO) and that methods that promote local search behaviors are better suited for the task. We find that maximum likelihood estimation (MLE) of GP length scales suffices for state-of-the-art performance. Based on this, we propose a simple variant of MLE called MSR that leverages these findings to achieve state-of-the-art performance on a comprehensive set of real-world applications. We present targeted experiments to illustrate and confirm our findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。