首次证明约束期望提升算法的收敛速率,理论严谨。
Convergence Rates of Constrained Expected Improvement
- 基于RKHS假设分析约束期望提升的收敛性
- 平方指数核下收敛速率达O(t^(-1/2) log^(d+1)/2(t))
- 适用于追求理论保障的优化研究者
约束贝叶斯优化(CBO)在带约束的黑箱优化中表现优异。其中,约束期望提升(CEI)是常用方法,但其理论收敛速率尚未明确。本文通过分析其简单遗憾上界,证明:当目标函数f和约束函数c均属于再生核希尔伯特空间(RKHS)时,采用平方指数核的CEI收敛率为O(t^(-1/2) log^((d+1)/2)(t)),采用Matérn核(ν > 1/2)时为O(t^(-ν/(2ν+d)) log^(ν/(2ν+d))(t))。进一步,在目标函数服从高斯过程(GP)假设下,该收敛率以高概率成立。数值实验验证了理论结果。
原文摘要 · Abstract (English)
Constrained Bayesian optimization (CBO) methods have seen significant success in black-box optimization with constraints. One of the most commonly used CBO methods is the constrained expected improvement (CEI) algorithm. CEI is a natural extension of expected improvement (EI) when constraints are incorporated. However, the theoretical convergence rate of CEI has not been established. In this work, we study the convergence rate of CEI by analyzing its simple regret upper bound. First, we show that when the objective function $f$ and constraint function $c$ are assumed to each lie in a reproducing kernel Hilbert space (RKHS), CEI achieves the convergence rates of $\mathcal{O} \left(t^{-\frac{1}{2}}\log^{\frac{d+1}{2}}(t) \right) \ \text{and }\ \mathcal{O}\left(t^{\frac{-ν}{2ν+d}} \log^{\fracν{2ν+d}}(t)\right)$ for the commonly used squared exponential and Matérn kernels ($ν>\frac{1}{2}$), respectively. Second, we show that when $f$ is assumed to be sampled from Gaussian processes (GPs), CEI achieves similar convergence rates with a high probability. Numerical experiments are performed to validate the theoretical analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。