研究贝叶斯优化初始点数量对总成本的影响，发现存在最优值。
How Many Initial Points Does Bayesian Optimization Need?
- 通过实验发现初始点数与总成本呈倒U型关系
- 过少或过多初始点都会增加资源浪费，存在最佳平衡点
- 推荐使用多步前瞻或Thompson采样以降低敏感性
贝叶斯优化通常从一批 $n_0$ 个无信息的初始评估开始。$n_0$ 的选择仍主要依赖经验，我们实证观察到：总成本（随机初始点加贝叶斯优化迭代次数）在 $n_0$ 上呈倒U型，即初值过低或过高都会造成资源浪费。该权衡在最大似然估计、贝叶斯MCMC和精确高斯过程超参数下均成立，且对不同采集函数普遍适用。值得注意的是，Thompson采样在此表现例外，其总成本与简单遗憾基本不受 $n_0$ 影响，尽管实验中整体成本较高。我们将其归因于方差驱动的贝叶斯优化固有的边界问题：早期优化会优先探索超立方体边缘，随后才向内收敛。通过已知超参数的3D优化轨迹验证了该现象。最终建议：尽可能使用多步前瞻贝叶斯优化；否则若无法调节 $n_0$，应选用Thompson采样；若可调节，则采用较大 $n_0$。
原文摘要 · Abstract (English)
Bayesian Optimization (BO) generally begins with an initialization phase: a batch of $n_0$ uninformed evaluations. The choice of $n_0$ remains largely heuristic, and we empirically observe that the total cost (random initial points plus BO iterations needed to find the global optimum) is U-shaped in $n_0$, i.e., a practitioner wastes resources by selecting either too low or too high a value of $n_0$. We find this tradeoff persists across MLE, Bayesian MCMC, and exact GP hyperparameters, as well as across acquisition functions. Toward the latter, Thompson Sampling appears an exception, with both total cost and simple regret essentially $n_0$-agnostic, though higher in our experiments. We attribute this U-shape to the known boundary issue of variance-driven BO: BO burns early budget on corners of the hypercube before turning inward. We demonstrate this effect using a 3D BO trajectory where the exact hyperparameters are known. We conclude with practical recommendations: use multi-step lookahead BO where possible; otherwise use Thompson Sampling when $n_0$ cannot be tuned, and a generously large $n_0$ when it can.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。