用高斯过程高效优化多目标问题,兼顾理论保障与实际效率。
Vector Optimization with Gaussian Process Bandits
- 基于高斯过程设计自适应淘汰算法,利用目标函数光滑性加速搜索。
- 平均样本复杂度降低约18倍,显著优于现有方法。
- 适合需要理论保证的高维多目标黑箱优化场景。
我们研究基于高斯过程带域的黑箱多目标优化问题,其中目标向量间存在由多面体凸锥描述的不完全序关系。现有黑箱多目标优化方法或样本复杂度高,或缺乏理论保证。本文提出向量优化高斯过程(VOGP),一种自适应淘汰算法,通过利用目标函数的光滑性,以高效样本识别帕累托最优解。我们建立了基于信息增益和核函数特性的理论样本复杂度边界。通过在多个真实世界和合成数据集上的全面实验评估,验证了VOGP的优越性:平均样本复杂度降低约18倍。此外,我们还提供了针对连续设计空间及核超参数未知情况的启发式改进方案。本工作在保持理论保证的同时,实现了多目标黑箱优化的样本高效新范式。
原文摘要 · Abstract (English)
We study black-box vector optimization with Gaussian process bandits, where there is an incomplete order relation on objective vectors described by a polyhedral convex cone. Existing black-box vector optimization approaches either suffer from high sample complexity or lack theoretical guarantees. We propose Vector Optimization with Gaussian Process (VOGP), an adaptive elimination algorithm that identifies Pareto optimal solutions sample efficiently by exploiting the smoothness of the objective function. We establish theoretical guarantees, deriving information gain-based and kernel-specific sample complexity bounds. Finally, we conduct a thorough empirical evaluation of VOGP and compare it with the state-of-the-art multi-objective and vector optimization algorithms on several real-world and synthetic datasets, emphasizing VOGP's efficiency (e.g., $\sim18\times$ lower sample complexity on average). We also provide heuristic adaptations of VOGP for cases where the design space is continuous and where the Gaussian process model lacks access to the true kernel hyperparameters. This work opens a new frontier in sample-efficient multi-objective black-box optimization by incorporating preference structures while maintaining theoretical guarantees and practical efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。