用变分自编码器降维,提升贝叶斯优化在高维问题上的效率与精度。
Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization
- 以VAE构建结构化低维隐空间,实现高维函数的高效采样优化。
- 结合隐空间序列域收缩(SDR-LSBO),优化质量显著优于传统随机投影。
- 适用于高维黑箱优化,尤其适合药物分子设计等复杂场景。
贝叶斯优化(BO)是高效全局优化昂贵黑箱函数的标准方法,但其在高维场景下的可扩展性仍面临挑战。本文研究非线性降维技术,将高维优化问题转化为一系列低维隐空间贝叶斯优化(LSBO)。相较于早期使用随机投影的方法,本文基于Grosnit等(2021)的工作,采用变分自编码器(VAE)进行LSBO,重点改进深度度量损失以构建结构化隐流形,并通过重训练编码器-解码器适应新采样区域。针对原有用于分子生成任务的VAE实现,提出适配优化目标的修改方案。进一步,直接在隐空间中耦合序列域收缩(SDR)策略,形成SDR-LSBO算法,随证据积累逐步缩小隐空间搜索范围。基于支持GPU加速的BoTorch框架,采用Matern-5/2高斯过程代理模型,实验表明该方法在多个基准任务上均获得更优优化质量,且结构化隐流形显著提升性能。对比随机嵌入与VAE降维,结果证明后者表现更优。据我们所知,这是首个将SDR与基于VAE的LSBO结合的研究,其分析明确了度量设计与重训练的关键作用,对可扩展隐空间优化具有指导意义。代码已开源:https://github.com/L-Lok/Nonlinear-Dimensionality-Reduction-Techniques-for-Bayesian-Optimization.git。
原文摘要 · Abstract (English)
Bayesian optimisation (BO) is a standard approach for sample-efficient global optimisation of expensive black-box functions, yet its scalability to high dimensions remains challenging. Here, we investigate nonlinear dimensionality reduction techniques that reduce the problem to a sequence of low-dimensional Latent-Space BO (LSBO). While early LSBO methods used (linear) random projections (Wang et al., 2013), building on Grosnit et al. (2021), we employ Variational Autoencoders (VAEs) for LSBO, focusing on deep metric loss for structured latent manifolds and VAE retraining to adapt the encoder-decoder to newly sampled regions. We propose some changes in their implementation, originally designed for tasks such as molecule generation, and reformulate the algorithm for broader optimisation purposes. We then couple LSBO with Sequential Domain Reduction (SDR) directly in the latent space (SDR-LSBO), yielding an algorithm that narrows the latent search domains as evidence accumulates. Implemented in a GPU-accelerated BoTorch stack with Matern-5/2 Gaussian process surrogates, our numerical results show improved optimisation quality across benchmark tasks and that structured latent manifolds improve BO performance. Additionally, we compare random embeddings and VAEs as two mechanisms for dimensionality reduction, showing that the latter outperforms the former. To the best of our knowledge, this is the first study to combine SDR with VAE-based LSBO, and our analysis clarifies design choices for metric shaping and retraining that are critical for scalable latent space BO. For reproducibility, our source code is available at https://github.com/L-Lok/Nonlinear-Dimensionality-Reduction-Techniques-for-Bayesian-Optimization.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。