提出新框架,无需梯度即可在复杂游戏中高效找局部鞍点。
A Framework for Finding Local Saddle Points in Two-Player Zero-Sum Black-Box Games
- 用高斯过程建模未知目标函数,仅需零阶采样。
- 在非凸非凹场景下,算法可快速定位局部鞍点。
- 适合无梯度、高成本评估的博弈优化任务。
鞍点优化是投资组合优化、生成对抗网络和机器人学等众多实际应用中的关键问题。传统方法多假设目标函数已知且可微,而黑箱场景中目标未知,仅能通过采样获得。现有工作或假设目标为凸-凹以简化问题,或依赖有噪声的梯度估计。本文提出一种受贝叶斯优化启发的框架,利用高斯过程建模未知(可能非凸-非凹)的目标函数,仅需零阶采样。该框架将鞍点优化视为两级过程:上层通过在有希望的区域采样构建目标函数模型;下层利用该模型将问题转化为广义博弈,并确定采样位置。下层设计可灵活调整,我们展示了不同变体在运行时间、评估成本和初始样本数之间的权衡。实验在合成及真实数据集上验证了该方法在黑箱非凸非凹设置下的有效性,证明其能高效定位局部鞍点。
原文摘要 · Abstract (English)
Saddle point optimization is a critical problem employed in numerous real-world applications, including portfolio optimization, generative adversarial networks, and robotics. It has been extensively studied in cases where the objective function is known and differentiable. Existing work in black-box settings with unknown objectives that can only be sampled either assumes convexity-concavity in the objective to simplify the problem or operates with noisy gradient estimators. In contrast, we introduce a framework inspired by Bayesian optimization which utilizes Gaussian processes to model the unknown (potentially nonconvex-nonconcave) objective and requires only zeroth-order samples. Our approach frames the saddle point optimization problem as a two-level process which can flexibly integrate existing and novel approaches to this problem. The upper level of our framework produces a model of the objective function by sampling in promising locations, and the lower level of our framework uses the existing model to frame and solve a general-sum game to identify locations to sample. This lower level procedure can be designed in complementary ways, and we demonstrate the flexibility of our approach by introducing variants which appropriately trade off between factors like runtime, the cost of function evaluations, and the number of available initial samples. We experimentally demonstrate these algorithms on synthetic and realistic datasets in black-box nonconvex-nonconcave settings, showcasing their ability to efficiently locate local saddle points in these contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。