arXiv:2605.23473cs.LGcs.AI2026-05中稿 · IJCAI

自动调整嵌入维度,让贝叶斯优化在高维未知场景下更高效。

Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension

论文配图:Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension
图 1 · 摘自论文原文
  • 动态调整低维子空间维度,根据解的质量决定是否升级。
  • 在多种高维函数和真实任务上,优化误差和耗时均显著降低。
  • 适合高维黑箱优化且有效维度未知的场景,如超参调优。

贝叶斯优化广泛用于优化复杂黑箱函数,但面临维数灾难问题。随机嵌入作为一种降维策略,可通过在低维子空间中优化来简化具有有效维度的任务。然而,预先确定任务的有效维度仍具挑战性,影响子空间维度选择与优化性能。传统方法依赖专家设定固定维度或通过试错估计,消耗大量资源。为此,本文提出一种面向高维贝叶斯优化且有效维度未知的自动化随机嵌入方法——动态共享嵌入贝叶斯优化(DSEBO)。DSEBO从低维开始,若当前子空间中的解显示初步收敛,则切换至更高维子空间。其根据不同子空间解的质量动态决定下一子空间的维度,并将已查询解共享给新子空间以实现更好初始化。理论上,我们推导了DSEBO的后悔界,证明其能更好平衡近似误差与优化误差。大量实验表明,在不同量级维度的函数及有效维度未知的真实任务上,相较于现有最优方法,跨子空间交替优化显著提升高维优化性能,无论是优化遗憾还是时间效率均表现优异。

原文摘要 · Abstract (English)

Bayesian optimization is widely employed for optimizing complex black-box functions but struggles with the curse of dimensionality. Random embedding, as a dimension reduction strategy, simplifies tasks that possess the effective dimension by optimizing within a low-dimensional subspace. However, determining the effective dimension of a task in advance remains a significant challenge, which influences the selection of the subspace dimensionality and the optimization performance. Traditional methods use fixed subspace dimensions provided by experts or rely on trial and error to estimate subspace dimensions with resources consumed. To this end, this paper proposes an automated random embedding for high-dimensional Bayesian optimization with unknown effective dimension, called Dynamic Shared Embedding Bayesian Optimization (DSEBO). DSEBO starts with a low dimension and switches to a higher subspace if the solutions in the current subspace show preliminary convergence. DSEBO dynamically determines the dimension of the next subspace based on the quality of the solutions in different subspaces and shares the queried solutions with the new subspace for a better initialization. Theoretically, we derive a regret bound for DSEBO and demonstrate that DSEBO can better balance approximation and optimization errors. Extensive experiments on functions with dimensionality of varying magnitudes and real-world tasks with unknown effective dimensions reveal that, compared with state-of-the-art methods, alternating optimization across different subspaces results in significant improvements in high-dimensional optimization, both in terms of optimization regret and time.

贝叶斯优化高维优化自动调参

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。