用神经网络加速贝叶斯优化中的熵搜索，单次前向传播即可完成评估。
$α$-PFN: Fast Entropy Search via In-Context Learning
- 分两阶段训练神经网络，用先验数据拟合网络逼近熵搜索目标。
- 在多个测试中实现超50倍加速，性能媲美当前最优方法。
- 适合需要快速迭代的黑箱优化任务，如超参调优与实验设计。
基于信息论的采集函数（如熵搜索）为贝叶斯优化提供了严谨的探索-利用框架。然而其实际应用依赖复杂的慢速近似方法，例如蒙特卡洛估计信息增益，易引入数值误差且需人工定制实现。本文提出一种两阶段摊销策略，通过先验-数据拟合网络（PFN）在单次前向传播中学习逼近熵搜索类采集函数。首先训练一个PFN以条件化于最优值信息；其次，训练α-PFN通过学习第一阶段PFN测量的信息增益来预测期望信息增益。α-PFN提供灵活的可学习近似，将复杂启发式近似替换为每候选点一次前向传播，实现快速且可扩展的采集函数评估。实验证明，该方法在合成与真实世界基准上性能媲美当前最优熵搜索实现，同时在所有实验中显著加速各类熵搜索变体，速度提升超过50倍。源码：https://github.com/automl/AlphaPFN。
原文摘要 · Abstract (English)
Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO). However, their practical implementation relies on complicated and slow approximations, i.e., a Monte Carlo estimation of the information gain. This complexity can introduce numerical errors and requires specialized, hand-crafted implementations. We propose a two-stage amortization strategy that learns to approximate entropy search-based acquisition functions using Prior-data Fitted Networks (PFNs) in a single forward pass. A first PFN is trained to be conditioned on information about the optima; second, the $α$-PFN is trained to predict the expected information gain by training on information gains measured with the first PFN. The $α$-PFN offers a flexible learned approximation, which replaces the complex heuristic approximations with a single forward pass per candidate, enabling rapid and extensible acquisition evaluation. Empirically, our approach is competitive with state-of-the-art entropy search implementations on synthetic and real-world benchmarks, while accelerating the different entropy search variants across all our experiments, with speed ups over 50x. Source code: https://github.com/automl/AlphaPFN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。