用核集成与分歧感知策略,提升贝叶斯优化与主动学习效率和精度。
Enhancing Bayesian Optimization and Active Learning Through Kernel Diversity

- 用核集成替代超参数采样,结合自适应加权与分歧感知选择
- 优化速度比顶尖方法快5倍,主动学习提速27倍且校准更准
- 统一框架适用于单目标、多目标及主动学习,适合高效建模场景
超参数选择仍是贝叶斯优化(BO)和贝叶斯主动学习(AL)中的关键挑战,模型误设会导致性能下降,而更精确的全贝叶斯方法通常依赖计算开销大的MCMC采样。本文提出统一框架KENDO(Kernel ENsemble Disagreement-aware Operator),将集成高斯过程(EGP)与分歧感知采集策略相结合。核心思想是用核集成与自适应贝叶斯加权替代超参数采样,并结合分歧感知采集策略。在此框架下,我们实现了KENDO-BO(用于BO)和KENDO-AL(用于AL),二者均源于共同的自校正机制,但针对任务设计不同采集目标。进一步通过随机标量化扩展至多目标优化,保持单优化器条件结构。在合成与真实世界基准上的全面测试表明:(i) KENDO-BO 在性能上媲美或优于最先进方法,同时计算开销降低高达5倍;(ii) KENDO-AL 在预测校准上优于基于MCMC的主动学习基线,最快达27倍加速。
原文摘要 · Abstract (English)
Hyperparameter selection remains a key challenge in Bayesian optimization (BO) and Bayesian active learning (AL), as model misspecification can lead to suboptimal performance, while more accurate fully Bayesian treatments typically rely on computationally expensive MCMC sampling. This paper proposes a unified framework, KENDO (Kernel ENsemble Disagreement-aware Operator), that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies. The central idea is to replace hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, combined with disagreement-aware acquisition strategies. Within this unified framework, we instantiate KENDO-BO for BO and KENDO-AL for Bayesian AL, demonstrating that both arise from a common self-correcting mechanism with task-specific acquisition objectives. We further extend the approach to multi-objective optimization via random scalarization that preserves the single-optimizer conditioning structure. Thorough numerical tests on synthetic and real-world benchmarks across single-objective optimization, multi-objective optimization, and active learning demonstrate that (i) KENDO-BO achieves competitive or superior optimization performance compared to state-of-the-art methods while reducing computational overhead by up to $5\times$ and (ii) KENDO-AL achieves superior predictive calibration over MCMC-based active learning baselines with up to $27\times$ speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。