arXiv:2412.15863cs.LG2024-12AAAI

新算法让贝叶斯优化在未知成本下仍能高效选变量,省钱又精准。

Bayesian Optimization for Unknown Cost-Varying Variable Subsets with No-Regret Costs

  • 分探索与利用两阶段,自动筛选低成本优质变量组合。
  • 理论证明质量与成本的后悔率均呈次线性增长,优于以往方法。
  • 适合高成本变量选择场景,如实验设计、资源受限优化。

贝叶斯优化(BO)常用于优化昂贵的黑箱函数。传统方法假设可完全控制所有查询变量,无额外约束。但在现实场景中,控制某些变量可能产生成本。因此,需权衡选择信息量高的变量子集与随机采样以降低成本。该问题称为带成本可变变量子集的贝叶斯优化(BOCVS)。以往工作虽保证找到最优解,却未考虑总成本;且假设知晓各子集的确切成本,这通常不现实。本文提出一种新算法,适用于随机且未知成本的BOCVS扩展问题。算法将过程分为探索与利用阶段:探索阶段剔除低质量变量子集,利用阶段聚焦高质量子集。我们理论证明该算法在质量与成本两个方面的后悔率均为次线性,更有效地解决了BOCVS目标。实验表明,该算法在多种基准测试中均优于可比基线。

原文摘要 · Abstract (English)

Bayesian Optimization (BO) is a widely-used method for optimizing expensive-to-evaluate black-box functions. Traditional BO assumes that the learner has full control over all query variables without additional constraints. However, in many real-world scenarios, controlling certain query variables may incur costs. Therefore, the learner needs to balance the selection of informative subsets for targeted learning against leaving some variables to be randomly sampled to minimize costs. This problem is known as Bayesian Optimization with cost-varying variable subsets (BOCVS). While the goal of BOCVS is to identify the optimal solution with minimal cost, previous works have only guaranteed finding the optimal solution without considering the total costs incurred. Moreover, these works assume precise knowledge of the cost for each subset, which is often unrealistic. In this paper, we propose a novel algorithm for the extension of the BOCVS problem with random and unknown costs that separates the process into exploration and exploitation phases. The exploration phase will filter out low-quality variable subsets, while the exploitation phase will leverage high-quality ones. Furthermore, we theoretically demonstrate that our algorithm achieves a sub-linear rate in both quality regret and cost regret, addressing the objective of the BOCVS problem more effectively than previous analyses. Finally, we show that our proposed algorithm outperforms comparable baselines across a wide range of benchmarks.

贝叶斯优化成本控制随机成本次线性后悔

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。