用梯度信息选样本，让贝叶斯优化更快更省资源
Gradient-based Sample Selection for Faster Bayesian Optimization
- 基于梯度筛选关键样本，减少冗余同时保持多样性
- 实验显示计算成本大幅降低，性能接近传统方法
- 适合大规模黑箱优化场景，尤其对算力敏感的任务
贝叶斯优化（BO）是求解黑箱优化问题的有效方法，但其应用通常受限于中等预算，主要因高斯过程（GP）代理模型的立方复杂度。在大预算情况下，直接使用标准GP模型面临严重的计算时间和资源消耗问题。本文提出一种新方法——基于梯度的样本选择贝叶斯优化（GSSBO），将GP模型构建在精选样本集上而非全数据集。通过利用梯度信息去除冗余，同时保留样本的多样性和代表性。我们提供了梯度样本选择策略的理论分析，并获得了所提框架的显式次线性遗憾界。在合成与真实任务上的大量实验表明，该方法显著降低了GP拟合的计算成本，同时保持了与基线方法相当的优化性能。
原文摘要 · Abstract (English)
Bayesian optimization (BO) is an effective technique for black-box optimization. However, its applicability is typically limited to moderate-budget problems due to the cubic complexity of fitting the Gaussian process (GP) surrogate model. In large-budget scenarios, directly employing the standard GP model faces significant challenges in computational time and resource requirements. In this paper, we propose a novel approach, gradient-based sample selection Bayesian Optimization (GSSBO), to enhance the computational efficiency of BO. The GP model is constructed on a selected set of samples instead of the whole dataset. These samples are selected by leveraging gradient information to remove redundancy while preserving diversity and representativeness. We provide a theoretical analysis of the gradient-based sample selection strategy and obtain explicit sublinear regret bounds for our proposed framework. Extensive experiments on synthetic and real-world tasks demonstrate that our approach significantly reduces the computational cost of GP fitting in BO while maintaining optimization performance comparable to baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。