用扩散模型加速贝叶斯优化，更准更快找全局最优。
Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
- 用条件扩散模型高效逼近最优解分布
- 提出新策略DMS，在10个基准上优于传统方法
- 适合需要高效优化的机器学习与工程场景
贝叶斯优化（BO）是一种广泛用于黑箱优化的方法，使用高斯过程（GP）作为代理模型，并通过采集函数指导序列评估以寻找全局最优解 $$\mathbf{x}^{∗}\u0024$。为实现此目标，基于信息的采集函数如预测熵搜索（PES）将 $$\mathbf{x}^{∗}\u0024$ 建模为随机变量并减少其分布的熵，但传统GP后验采样计算成本高昂。为此，本文利用条件扩散模型（CDM）高效近似 $$\mathbf{x}^{∗}\u0024$ 的分布，并设计了面向BO的训练策略。基于CDM学习分布的结构特性，进一步提出一种名为扩散模式搜索（DMS）的采集策略以引导序列评估。本文建立了CDM学习分布的次优性保证，并通过大量实验表明，DMS在多个基准测试中超越标准BO基线。
原文摘要 · Abstract (English)
Bayesian optimization (BO) is a widely used approach for black-box optimization that uses a Gaussian process (GP) as a surrogate and guides sequential evaluations via an acquisition function, with the ultimate goal of locating the global optimum $\mathbf{x}^{\star}$. To align with this goal, information-based acquisition functions such as Predictive Entropy Search (PES) model $\mathbf{x}^{\star}$ as a random variable and reduce the entropy of its distribution, but approximating this distribution via traditional GP posterior sampling is computationally expensive. To address this limitation, we leverage Conditional Diffusion Models (CDMs) to efficiently approximate the distribution of $\mathbf{x}^{\star}$ and develop BO-inherent training strategies for CDMs. Motivated by the structural properties of the CDM-learned distribution, we further develop an acquisition strategy termed Diffusion-based Mode Seeking (DMS) to guide the sequential evaluation. We establish a sub-optimality guarantee for the CDM-learned distribution and demonstrate through extensive experiments that DMS outperforms standard BO baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。