提出分步优化加速贝叶斯优化，提升效率且收敛性不变。
Batch Acquisition Function Evaluations and Decouple Optimizer Updates for Faster Bayesian Optimization
- 批处理评估+协程解耦优化器更新，避免矩阵近似误差。
- 相比传统方法，速度显著提升，收敛性与串行优化一致。
- 已集成至Optuna的GPSampler，适合大规模超参优化场景。
贝叶斯优化通过最大化采集函数高效寻找高性能参数，但其优化过程因采集函数非凸性需采用多起点优化（MSO）与拟牛顿（QN）方法，计算开销大。当前主流库BoTorch通过PyTorch批处理对多个点同时优化总和采集函数以提速，但本文实证表明该方法在拟牛顿法逆海森矩阵的非对角项近似上存在次优性，阻碍收敛。为此，本文提出在批处理采集函数调用的同时，使用协程解耦优化器更新，使算法理论收敛性等同于串行多起点优化，并大幅降低实际运行时间。该方法已集成至Optuna的GPSampler中，有效减少计算开销。
原文摘要 · Abstract (English)
Bayesian optimization (BO) efficiently finds high-performing parameters by maximizing an acquisition function, which models the promise of parameters. A major computational bottleneck arises in acquisition function optimization, where multi-start optimization (MSO) with quasi-Newton (QN) methods is required due to the non-convexity of the acquisition function. BoTorch, a widely used BO library, currently optimizes the summed acquisition function over multiple points, leading to the speedup of MSO owing to PyTorch batching. Nevertheless, this paper empirically demonstrates the suboptimality of this approach in terms of off-diagonal approximation errors in the inverse Hessian of a QN method, slowing down its convergence. To address this problem, we propose to decouple QN updates using a coroutine while batching the acquisition function calls. Our approach not only yields the theoretically identical convergence to the sequential MSO but also drastically reduces the wall-clock time compared to the previous approaches. Our approach is available in GPSampler in Optuna, effectively reducing its computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。