用有限映射库动态调整输入空间,让贝叶斯优化更快更准。
No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

- 从有限光滑映射库中选输入变换,自适应调整输入几何
- 在4个基准上比原始方法快1.5~3倍,且有理论收敛保证
- 适合超参调优、复杂系统设计等高成本优化场景
高斯过程贝叶斯优化(GP-BO)在昂贵函数优化中表现优异,如超参数优化(HPO)和多智能体系统(MAS)设计。现有方法如GP-UCB虽有收敛率保证,但需固定核函数。核函数决定了输入距离如何影响目标值相似性。当原始坐标与真实几何不匹配时(如对数缩放超参或局部峰值),输入扭曲可显著提升样本效率,但已有证明要求核函数固定。本文提出有限库输入扭曲贝叶斯优化(FLIWBO),通过任意历史依赖规则从有限光滑映射库中选择扭曲方式,在保持高概率收敛性的同时适应输入几何,其代价为显式的√(N_ε)库大小项。控制诊断显示,有限库扭曲能修复预设的几何错配,并识别失败情况。在四个重复基准测试中——扭曲合成目标、置信区间陷阱、Fashion-MNIST HPO——FLIWBO-UCB在几何误设下优于原始坐标GP-UCB,逃脱了即使使用最优扭曲期望改进也失败的陷阱,并恢复了手动对数缩放的大部分优势;同时领先于所有具备匹配后悔保证的对比方法。20维MAS设计研究进一步验证了其在高成本噪声评估下的可行性。实验代码已公开:https://github.com/edvin-ketabati/bogp-paper-experiments。
原文摘要 · Abstract (English)
Gaussian-process Bayesian optimization (GP-BO) excels at black-box optimization of costly functions, e.g., hyperparameter optimization (HPO) and multi-agent system (MAS) design. Convergence-rate guarantees exist for select methods, notably GP upper confidence bound (GP-UCB), but require a fixed kernel. Critically, the kernel encodes how input proximity affects objective value similarity. When raw coordinates poorly match this geometry - as with log-scaled hyperparameters or localized peaks - input warping can greatly improve sample efficiency, yet known GP-UCB proofs require a fixed kernel. We propose Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects warps from a finite library of smooth input maps by any history-dependent rule. It adapts the input geometry to accelerate learning while retaining high-probability convergence guarantees under mild hypotheses, with an explicit $\sqrt(N_\varepsilon)$ library-size cost. Controlled diagnostics show that finite-library warping repairs planted geometry mismatches and identify FLIWBO failure cases. Across four repeated benchmarks - warped synthetic objectives, a confidence-fence trap, and Fashion-MNIST HPO - FLIWBO-UCB beats raw-coordinate GP-UCB under misspecified geometry, escapes traps that defeat even oracle-warp expected improvement, and recovers much of the gain from manual log scaling, while leading the tested methods that admit a matching regret guarantee. A 20-dimensional MAS design study further shows feasibility under costly noisy evaluations. Code for experiments is available: https://github.com/edvin-ketabati/bogp-paper-experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。