arXiv:2505.11281stat.MLcs.LG2025-05

让优化算法自动切换低维投影,应对高维非平稳问题。

Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization

  • 用多个随机投影捕捉不同局部结构,动态选择适合的投影方式。
  • 在真实和合成测试中显著优于传统方法,有效处理高维非平稳目标。
  • 适合复杂设计空间中的高维优化,尤其适用于有效维度变化大的场景。

高维贝叶斯优化受限于维度诅咒和全局低维假设的刚性。尽管随机嵌入贝叶斯优化(REMBO)通过线性投影降低维度缓解此问题,但通常假设单一全局嵌入且目标平稳。本文提出自适应嵌入REMBO(SA-REMBO),支持多个随机高斯嵌入,分别捕捉高维目标的不同局部子空间结构。一个索引变量控制嵌入选择,并与潜在优化变量共同通过乘积核建模于高斯过程代理模型中。该机制使优化器能根据位置自适应选择嵌入,有效捕获局部变化的有效维度、非平稳性和异方差性。我们理论分析了索引条件乘积核的表达能力与稳定性,并在合成及真实世界高维基准测试中实证验证了方法优势,传统REMBO及其他低秩贝叶斯优化方法在此类任务中均表现不佳。结果表明,SA-REMBO是复杂、结构化设计空间中可扩展贝叶斯优化的强大且灵活的拓展。

原文摘要 · Abstract (English)

Bayesian Optimization (BO) in high-dimensional spaces remains fundamentally limited by the curse of dimensionality and the rigidity of global low-dimensional assumptions. While Random EMbedding Bayesian Optimization (REMBO) mitigates this via linear projections into low-dimensional subspaces, it typically assumes a single global embedding and a stationary objective. In this work, we introduce Self-Adaptive embedding REMBO (SA-REMBO), a novel framework that generalizes REMBO to support multiple random Gaussian embeddings, each capturing a different local subspace structure of the high-dimensional objective. An index variable governs the embedding choice and is jointly modeled with the latent optimization variable via a product kernel in a Gaussian Process surrogate. This enables the optimizer to adaptively select embeddings conditioned on location, effectively capturing locally varying effective dimensionality, nonstationarity, and heteroscedasticity in the objective landscape. We theoretically analyze the expressiveness and stability of the index-conditioned product kernel and empirically demonstrate the advantage of our method across synthetic and real-world high-dimensional benchmarks, where traditional REMBO and other low-rank BO methods fail. Our results establish SA-REMBO as a powerful and flexible extension for scalable BO in complex, structured design spaces.

贝叶斯优化高维优化自适应嵌入非平稳性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。