提出一种自适应成本的贝叶斯优化停止策略,兼顾效率与精度。
Cost-aware Stopping for Bayesian Optimization
- 基于成本感知的采集函数设计停机规则,无需手动调参。
- 在合成任务和真实场景中,成本调整后的误差显著优于现有方法。
- 适合高成本黑箱优化,如超参数调优与神经网络结构搜索。
在自动化机器学习、科学发现等贝叶斯优化应用中,如何以成本意识决定何时停止昂贵的黑箱函数评估是一个重要但未充分探索的问题。一个自然的评价指标是成本调整后的简单遗憾,它明确捕捉了解决方案质量与累计评估成本之间的权衡。现有停止规则要么是启发式方法,要么虽有理论基础但未考虑评估成本,因此在成本较高时无法保证避免不必要的评估。我们提出一种原则性的成本感知停止规则,可自适应不同评估成本而无需启发式调参。该规则基于与先进成本感知采集函数(即潘多拉之盒吉廷斯指数(PBGI)和单位成本对数期望改进(LogEIPC))的理论关联。当与任一采集函数结合时,我们证明了其策略能提供预期成本调整后简单遗憾的理论界。在合成任务及包括超参数优化和神经架构规模搜索在内的实证基准上,该停止规则与PBGI或LogEIPC配合,通常在成本调整后简单遗憾方面表现匹配或优于其他采集函数-停止规则组合。
原文摘要 · Abstract (English)
In automated machine learning, scientific discovery, and other applications of Bayesian optimization, deciding when to stop evaluating expensive black-box functions in a cost-aware manner is an important but underexplored practical consideration. A natural performance metric for this purpose is the cost-adjusted simple regret, which explicitly captures the trade-off between solution quality and cumulative evaluation cost. Existing stopping rules for Bayesian optimization are either heuristic, or are theoretically grounded but designed to optimize simple regret without accounting for evaluation costs; as a result, they provide no guarantees against unnecessary evaluations when costs are high. We propose a principled cost-aware stopping rule for Bayesian optimization that adapts to varying evaluation costs without heuristic tuning. Our rule is grounded in a theoretical connection to state-of-the-art cost-aware acquisition functions, namely the Pandora's Box Gittins Index (PBGI) and log expected improvement per cost (LogEIPC). When paired with either acquisition function, we prove that the resulting policy satisfies a theoretical guarantee bounding the expected cost-adjusted simple regret. Across synthetic tasks and empirical benchmarks including hyperparameter optimization and neural architecture size search, pairing our stopping rule with PBGI or LogEIPC usually matches or outperforms other acquisition-function--stopping-rule pairs in terms of cost-adjusted simple regret.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。