通过随机投影与几何感知学习,实现高维黑箱优化的高效求解。
High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces
- 用随机线性投影降维,结合非线性流形表示学习。
- 在合成函数与真实场景中均优于现有高维贝叶斯优化方法。
- 适合数据有限、维度高且存在低维结构的优化任务。
贝叶斯优化(BO)是优化昂贵黑箱函数的常用方法,但其性能随维度增加呈指数下降。常见策略假设目标函数仅依赖于嵌入高维空间的低维流形上的少数特征。现有方法多采用编码器-解码器框架学习特征映射,但计算成本高或在标注数据少时易过拟合。本文提出一种新方法:结合随机线性投影降低维度,并学习非线性流形的表示。当流形几何信息已知时,设计几何感知的表示学习方案;否则使用神经网络,并通过几何感知的半监督方式训练以缓解过拟合。该方法可在低维空间高效优化采集函数,并能准确映射回原高维空间。实验表明,该算法在多种合成函数及真实应用中均优于其他高维贝叶斯优化基线。
原文摘要 · Abstract (English)
Bayesian Optimization (BO) is a popular approach to optimizing expensive-to-evaluate black-box functions. Despite the success of BO, its performance may decrease exponentially as the dimensionality increases. A common framework to tackle this problem is to assume that the objective function depends on a limited set of features that lie on a low-dimensional manifold embedded in the high-dimensional ambient space. The latent space can be linear or more generally nonlinear. To learn feature mapping, existing works usually use an encode-decoder framework which is either computationally expensive or susceptible to overfittting when the labeled data is limited. This paper proposes a new approach for BO in high dimensions by exploiting a new representation of the objective function. Our approach combines a random linear projection to reduce the dimensionality, with a representation learning of the nonlinear manifold. When the geometry of the latent manifold is available, a solution to exploit this geometry is proposed for representation learning. In contrast, we use a neural network. To mitigate overfitting by using the neural network, we train the feature mapping in a geometry-aware semi-supervised manner. Our approach enables efficient optimizing of BO's acquisition function in the low-dimensional space, with the advantage of projecting back to the original high-dimensional space compared to existing works in the same setting. Finally, we show empirically that our algorithm outperforms other high-dimensional BO baselines in various synthetic functions and real applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。