arXiv:2504.07388math.OCcs.AI2025-04被引 5

提出随机零阶外梯度算法,解决非凸非凹极小极大问题的收敛性难题。

Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithm

  • 用随机高斯平滑构造零阶外梯度,处理不可导目标函数。
  • 在无约束与约束情形下,均证明算法收敛至ε-驻点邻域。
  • 适用于黑箱优化场景,适合不依赖梯度的极小极大学习任务。

本文研究随机高斯平滑零阶外梯度(ZO-EG)算法在可能非凸非凹(NC-NC)目标函数下的性能,涵盖无约束与有约束、可微与不可微情形。从变分不等式角度分析极小极大问题。对于无约束情形,证明了算法收敛至原目标函数ε-驻点的邻域,且邻域半径可通过方差缩减控制,并给出复杂度分析。针对有约束情形,提出新的近端变分不等式概念,并给出满足该性质的函数示例,同时获得与无约束情形类似的结果。对于不可微情形,证明算法收敛至平滑后目标函数ε-驻点邻域,其邻域半径可调控,该结果可关联到原函数的(δ,ε)-Goldstein驻点。

原文摘要 · Abstract (English)

This study explores the performance of the random Gaussian smoothing Zeroth-Order ExtraGradient (ZO-EG) scheme considering \Af{deterministic} min-max optimisation problems with possibly NonConvex-NonConcave (NC-NC) objective functions. We consider both unconstrained and constrained, differentiable and non-differentiable settings. We discuss the min-max problem from the point of view of variational inequalities. For the unconstrained problem, we establish the convergence of the ZO-EG algorithm to the neighbourhood of an $ε$-stationary point of the NC-NC objective function, whose radius can be controlled under a variance reduction scheme, along with its complexity. For the constrained problem, we introduce the new notion of proximal variational inequalities and give examples of functions satisfying this property. Moreover, we prove analogous results to the unconstrained case for the constrained problem. For the non-differentiable case, we prove the convergence of the ZO-EG algorithm to a neighbourhood of an $ε$-stationary point of the smoothed version of the objective function, where the radius of the neighbourhood can be controlled, which can be related to the ($δ,ε$)-Goldstein stationary point of the original objective function.

极小极大零阶优化非凸非凹随机算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。