通过知识迁移加速贝叶斯优化，理论证明可显著降低误差累积。
Provable Accelerated Bayesian Optimization with Knowledge Transfer
- 构建源-目标函数差异的不确定性量化机制，允许不同核空间。
- 理论证明误差为 $\widetilde{O}(\sqrt{T (T/N + γ_δ)})$，优于基线。
- 适用于超参数调优等相似任务场景，实测性能领先。
我们研究如何通过从相关源任务迁移历史知识来加速目标任务的贝叶斯优化（BO）。现有方法要么缺乏理论保证，要么达到与非迁移设置下相同的遗憾界 $\widetilde{O}(\sqrt{T γ_f})$，其中 $T$ 是目标函数评估次数，$γ_f$ 表示其信息增益。本文提出 DeltaBO 算法，对源与目标函数的差值函数 $δ$ 建立新颖的不确定性量化方法，允许两者属于不同的再生核希尔伯特空间（RKHS）。在合理假设下，证明 DeltaBO 的遗憾为 $\widetilde{O}(\sqrt{T (T/N + γ_δ)})$，其中 $N$ 为源任务评估次数，通常 $N \gg T$。在多数实际应用中，源与目标任务相似，导致 $γ_δ$ 远小于 $γ_f$。真实世界超参数调优任务与合成函数上的实验表明，DeltaBO 显著优于其他基线方法，并验证了理论预测。代码已开源。
原文摘要 · Abstract (English)
We study how to accelerate Bayesian optimization (BO) on a target task by transferring historical knowledge from related source tasks. Existing work on BO with knowledge transfer either lacks theoretical guarantees or achieves the same regret as BO in the non-transfer setting, $\widetilde{O}(\sqrt{T γ_f})$, where $T$ is the number of evaluations of the target function and $γ_f$ denotes its information gain. In this paper, we propose the DeltaBO algorithm, which builds a novel uncertainty-quantification approach on the difference function $δ$ between the source and target functions, which are allowed to belong to different Reproducing Kernel Hilbert Spaces (RKHSs). Under mild assumptions, we prove that the regret of DeltaBO is of order $\widetilde{O}(\sqrt{T (T/N + γ_δ)})$, where $N$ denotes the number of evaluations from source tasks and typically $N \gg T$. In many applications, source and target tasks are similar, which implies that $γ_δ$ can be much smaller than $γ_f$. Empirical studies on both real-world hyperparameter-tuning tasks and synthetic functions show that DeltaBO outperforms other baseline methods and also verify our theoretical claims. Our code is available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。