用预训练模型实现无需调参的高效黑箱优化
ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization
- 基于大规模优化轨迹和合成函数预训练，学习通用优化策略
- 零样本测试下在20维问题上超越主流优化器，样本效率更高
- 适合需要快速部署、无需调参的工程优化场景
昂贵且无导数的黑箱函数全局优化需要极高的样本效率。尽管贝叶斯优化（BO）是当前最先进方法，但其性能依赖于代理模型和采集函数的超参数，通常需手动调优且难以跨问题泛化。我们提出 ZeroShotOpt，一种适用于2至20维连续黑箱优化任务的通用预训练模型。该方法在12种不同贝叶斯优化变体生成的大规模优化轨迹上进行离线强化学习。为提升预训练规模，我们生成了数百万个具有多样化景观的合成高斯过程函数，使模型能够学习可迁移的优化策略。结果表明，ZeroShotOpt在大量未见过的基准测试中实现稳健的零样本泛化，在样本效率上匹配甚至超越领先的全局优化器，同时为未来改进提供可复用的基础。代码、数据集与模型已开源：https://github.com/jamisonmeindl/zeroshotopt
原文摘要 · Abstract (English)
Global optimization of expensive, derivative-free black-box functions requires extreme sample efficiency. While Bayesian optimization (BO) is the current state-of-the-art, its performance hinges on surrogate and acquisition function hyper-parameters that are often hand-tuned and fail to generalize across problem landscapes. We present ZeroShotOpt, a general-purpose, pretrained model for continuous black-box optimization tasks ranging from 2D to 20D. Our approach leverages offline reinforcement learning on large-scale optimization trajectories collected from 12 BO variants. To scale pretraining, we generate millions of synthetic Gaussian process-based functions with diverse landscapes, enabling the model to learn transferable optimization policies. As a result, ZeroShotOpt achieves robust zero-shot generalization on a wide array of unseen benchmarks, matching or surpassing the sample efficiency of leading global optimizers, including BO, while also offering a reusable foundation for future extensions and improvements. Our open-source code, dataset, and model are available at: https://github.com/jamisonmeindl/zeroshotopt
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。