用预训练模型直接采样最优解，实现无需调参的高效贝叶斯优化。
Simplifying Bayesian Optimization Via In-Context Direct Optimum Sampling
- 基于预训练生成模型直接从最优解后验中采样，跳过传统建模与优化步骤。
- 相比高斯过程贝叶斯优化，耗时减少35倍以上，支持并行与分布式计算。
- 零样本、无调参，适合需要快速部署的高通量优化场景。
昂贵黑箱函数的优化在科学与工程中极为普遍。传统贝叶斯优化（BO）通常包含两个组件：(i) 代理模型和 (ii) 获取函数，分别需要每次迭代时重新训练和优化，成本高昂。尽管近期工作实现了无需重训练的上下文代理模型，但几乎所有现有方法仍需对获取函数进行最大化以选择下一个观测点，涉及大量可调参数，如蒙特卡洛采样器和多起点优化器。本文提出一种完全上下文化、零样本的贝叶斯优化方案，无需代理拟合或获取函数优化。该方法利用预训练深度生成模型直接从最优点后验中采样。我们证明此过程等价于汤普森采样，并在一系列真实世界基准上验证了其性能与成本效益。相比基于高斯过程的贝叶斯优化，壁挂时长提升超过35倍，支持高效的并行与分布式优化，适用于高通量优化任务。
原文摘要 · Abstract (English)
The optimization of expensive black-box functions is ubiquitous in science and engineering. A common solution to this problem is Bayesian optimization (BO), which is generally comprised of two components: (i) a surrogate model and (ii) an acquisition function, which generally require expensive re-training and optimization steps at each iteration, respectively. Although recent work enabled in-context surrogate models that do not require re-training, virtually all existing BO methods still require acquisition function maximization to select the next observation, which introduces many knobs to tune, such as Monte Carlo samplers and multi-start optimizers. In this work, we propose a completely in-context, zero-shot solution for BO that does not require surrogate fitting or acquisition function optimization. This is done by using a pre-trained deep generative model to directly sample from the posterior over the optimum point. We show that this process is equivalent to Thompson sampling and demonstrate the capabilities and cost-effectiveness of our foundation model on a suite of real-world benchmarks. We achieve an efficiency gain of more than 35x in terms of wall-clock time when compared with Gaussian process-based BO, enabling efficient parallel and distributed BO, e.g., for high-throughput optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。