arXiv:2501.13332cs.LGstat.ML2025-01被引 7

用多个模型协作提升贝叶斯优化精度,避免陷入局部最优。

Co-Learning Bayesian Optimization

  • 构建多个高斯过程模型,通过多样性与一致性协同优化
  • 在5个数学测试和3个工程问题上显著优于传统方法
  • 适合样本有限、需高效全局优化的黑箱问题

贝叶斯优化(BO)在求解黑箱问题时以样本效率高著称,但常因高斯过程(GP)代理模型精度不足而陷入次优解。尤其在最优解附近区域,模型表现更差。为此,本文提出多模型协作的共学习贝叶斯优化(CLBO),通过构建多个互补的GP模型,同时利用模型间的一致性与多样性来提升整体代理模型准确性。基于径雷马赫复杂度理论,模型在未标记数据上的共识可降低假设空间复杂度,从而以更少样本达到所需精度。实验在五个数值玩具问题和三个工程基准上验证了该方法的有效性,结果表明其在有限样本下能实现更高效的全局优化。

原文摘要 · Abstract (English)

Bayesian optimization (BO) is well known to be sample-efficient for solving black-box problems. However, the BO algorithms can sometimes get stuck in suboptimal solutions even with plenty of samples. Intrinsically, such suboptimal problem of BO can attribute to the poor surrogate accuracy of the trained Gaussian process (GP), particularly that in the regions where the optimal solutions locate. Hence, we propose to build multiple GP models instead of a single GP surrogate to complement each other and thus resolving the suboptimal problem of BO. Nevertheless, according to the bias-variance tradeoff equation, the individual prediction errors can increase when increasing the diversity of models, which may lead to even worse overall surrogate accuracy. On the other hand, based on the theory of Rademacher complexity, it has been proved that exploiting the agreement of models on unlabeled information can help to reduce the complexity of the hypothesis space, and therefore achieving the required surrogate accuracy with fewer samples. Such value of model agreement has been extensively demonstrated for co-training style algorithms to boost model accuracy with a small portion of samples. Inspired by the above, we propose a novel BO algorithm labeled as co-learning BO (CLBO), which exploits both model diversity and agreement on unlabeled information to improve the overall surrogate accuracy with limited samples, and therefore achieving more efficient global optimization. Through tests on five numerical toy problems and three engineering benchmarks, the effectiveness of proposed CLBO has been well demonstrated.

贝叶斯优化高斯过程多模型协作全局优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。