用稀疏化方法选关键变量，高效优化高维黑箱函数
High Dimensional Bayesian Optimization using Lasso Variable Selection
- 通过高斯过程核的长度尺度估计识别重要变量
- 在多个子空间中优化，累积后悔值呈次线性增长
- 适合高维、样本稀缺的优化场景，计算高效
贝叶斯优化（BO）是优化昂贵黑箱函数的主流方法，但在高维场景下受维度诅咒影响，难以扩展。现有方法采用变量选择策略，迭代选取部分变量进行优化，虽缓解了高维问题，但仍存在样本效率低的问题。本文提出新方法：通过估计高斯过程核的长度尺度，识别重要变量；构建由多个子空间组成的有效搜索区域，并仅在该区域内优化采集函数，聚焦于关键变量。理论证明该方法在最坏情况下累积后悔值呈次线性增长，同时保持计算效率。在高维合成函数与真实世界问题上的实验表明，该方法性能达到当前最优水平。
原文摘要 · Abstract (English)
Bayesian optimization (BO) is a leading method for optimizing expensive black-box optimization and has been successfully applied across various scenarios. However, BO suffers from the curse of dimensionality, making it challenging to scale to high-dimensional problems. Existing work has adopted a variable selection strategy to select and optimize only a subset of variables iteratively. Although this approach can mitigate the high-dimensional challenge in BO, it still leads to sample inefficiency. To address this issue, we introduce a novel method that identifies important variables by estimating the length scales of Gaussian process kernels. Next, we construct an effective search region consisting of multiple subspaces and optimize the acquisition function within this region, focusing on only the important variables. We demonstrate that our proposed method achieves cumulative regret with a sublinear growth rate in the worst case while maintaining computational efficiency. Experiments on high-dimensional synthetic functions and real-world problems show that our method achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。