统一解释了贝叶斯优化中两种主流方法的内在联系。
A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization
- 提出变分熵搜索框架，揭示期望改进与信息理论方法本质关联。
- 新方法VES-Gamma在低维和高维测试中均优于或媲美现有方法。
- 适合对贝叶斯优化理论机制感兴趣的研究人员参考。
贝叶斯优化是优化昂贵黑箱函数的常用方法，期望改进（Expected Improvement, EI）是最常用的采集函数之一。相比之下，信息论型采集函数旨在减少对函数最优值的不确定性，常被视为与EI根本不同。本文挑战这一主流观点，提出统一的理论框架——变分熵搜索（Variational Entropy Search），揭示EI与信息论采集函数的密切关联。我们证明EI可被看作一种变分推断近似，对应于流行的极大值熵搜索（Max-value Entropy Search, MES）。基于此，提出新型采集函数VES-Gamma，融合了EI与MES的优势。在多种低维与高维合成及真实世界基准上的广泛实验表明，VES-Gamma表现优异，多数情况下超越或媲美当前最先进采集函数。
原文摘要 · Abstract (English)
Bayesian optimization is a widely used method for optimizing expensive black-box functions, with Expected Improvement being one of the most commonly used acquisition functions. In contrast, information-theoretic acquisition functions aim to reduce uncertainty about the function's optimum and are often considered fundamentally distinct from EI. In this work, we challenge this prevailing perspective by introducing a unified theoretical framework, Variational Entropy Search, which reveals that EI and information-theoretic acquisition functions are more closely related than previously recognized. We demonstrate that EI can be interpreted as a variational inference approximation of the popular information-theoretic acquisition function, named Max-value Entropy Search. Building on this insight, we propose VES-Gamma, a novel acquisition function that balances the strengths of EI and MES. Extensive empirical evaluations across both low- and high-dimensional synthetic and real-world benchmarks demonstrate that VES-Gamma is competitive with state-of-the-art acquisition functions and in many cases outperforms EI and MES.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。