首次给出非凸双层优化的联合驻点求解复杂度,为算法设计提供理论保障。
On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
- 提出非凸双层问题的联合驻点定义,用动态障碍梯度法实现可计算解。
- 算法在多项式时间内收敛至(ε_f, ε_g)驻点,复杂度为O(max(ε_f^(-(3+p)/(1+p)), ε_g^(-(3+p)/2)))。
- 适用于无凸性假设的通用非凸双层优化,适合关注理论保证的研究者。
本文研究一般非凸简单双层优化问题,其中上层目标函数在下层问题解集上最小化。上下层目标均为光滑但可能非凸,且不依赖下层目标的凸性或Polyak-Łojasiewicz条件。为此,我们引入适用于该类问题的合适驻点概念,并设计一种一阶算法以多项式时间找到此类驻点。直观上,驻点意味着上层目标无法在不显著恶化下层目标的情况下局部改进。我们证明,动态障碍梯度下降(DBGD)的一种简单可实现变体可有效解决这类问题至驻点。具体而言,达到(ε_f, ε_g)-驻点所需复杂度为\mathcal{O}\left(\max\left(ε_f^{-\frac{3+p}{1+p}}, ε_g^{-\frac{3+p}{2}}\right)\right),其中p ≥ 0为任意平衡参数。据我们所知,这是首个针对离散时间算法在一般非凸简单双层问题中保证双层联合驻点的复杂度结果。
原文摘要 · Abstract (English)
In this paper, we study the problem of solving a simple bilevel optimization problem, where the upper-level objective is minimized over the solution set of the lower-level problem. We focus on the general setting in which both the upper- and lower-level objectives are smooth but potentially nonconvex. Due to the absence of additional structural assumptions for the lower-level objective-such as convexity or the Polyak-Łojasiewicz (PL) condition-guaranteeing global optimality is generally intractable. Instead, we introduce a suitable notion of stationarity for this class of problems and aim to design a first-order algorithm that finds such stationary points in polynomial time. Intuitively, stationarity in this setting means the upper-level objective cannot be substantially improved locally without causing a larger deterioration in the lower-level objective. To this end, we show that a simple and implementable variant of the dynamic barrier gradient descent (DBGD) framework can effectively solve the considered nonconvex simple bilevel problems up to stationarity. Specifically, to reach an $(ε_f, ε_g)$-stationary point-where $ε_f$ and $ε_g$ denote the target stationarity accuracies for the upper- and lower-level objectives, respectively-the considered method achieves a complexity of $\mathcal{O}\left(\max\left(ε_f^{-\frac{3+p}{1+p}}, ε_g^{-\frac{3+p}{2}}\right)\right)$, where $p \geq 0$ is an arbitrary constant balancing the terms. To the best of our knowledge, this is the first complexity result for a discrete-time algorithm that guarantees joint stationarity for both levels in general nonconvex simple bilevel problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。