arXiv:2606.01764math.OCcs.GT2026-06

用幂律步长加速外梯度法,让收敛速度逼近最优

Accelerating Min-Max Optimization via Power-Law Stepsizes

  • 设计幂律动态步长,使外梯度法收敛率提升至 T^(-2/3+ε)
  • 不同步骤用不同步长时,收敛率达近似最优的 T^(-1+ε)
  • 方法可推广至乐观梯度等其他极小极大优化算法

我们重新研究了无约束双仿射极小极大优化中外梯度(EG)方法的收敛性。已知固定步长的EG方法最后迭代点收敛率为Θ(T^{-1/2}),慢于通过附加机制(如锚定)实现的最优O(T^{-1})速率。受近期动态步长能显著加速梯度下降的启发,我们探究动态步长是否也能加速EG的最后迭代收敛。本文首次给出正面结果:提出一种确定性动态步长调度,使EG收敛率提升至O(T^{-2/3+ε})(任意ε > 0)。我们还证明该速率在预估与更新步骤使用相同步长时是紧的。进一步地,若允许两步骤使用不同步长,收敛率可提升至近最优的O(T^{-1+ε})。我们的分析将步长调度转化为优化问题,其解对应离散化的幂律分布。所提步长策略及分析可扩展至乐观梯度(OG)等方法,并暗示对一般极小极大优化问题具有更广泛适用性。

原文摘要 · Abstract (English)

We revisit the convergence guarantees of the Extragradient (EG) method for unconstrained biaffine min-max optimization. It is known that EG with a fixed stepsize achieves a $Θ(T^{-1/2})$ last-iterate convergence rate, which is slower than the optimal $\mathcal{O}(T^{-1})$ rate attainable by incorporating additional mechanisms such as anchoring. Motivated by recent advances showing that dynamic stepsizes alone can significantly accelerate gradient descent, we ask whether dynamic stepsizes can similarly accelerate the last-iterate convergence of EG. We present the first positive result in this direction. Specifically, we provide a deterministic dynamic stepsize schedule that accelerates the convergence rate of EG to $\mathcal{O}(T^{-2/3+\varepsilon})$ for any $\varepsilon > 0$. We also show that this rate is tight when the extrapolation and update steps of EG use the same stepsize. We then show that allowing different stepsizes for the extrapolation and update steps further improves the convergence rate to the near-optimal $\mathcal{O}(T^{-1+\varepsilon})$. Our analysis reduces stepsize scheduling to an optimization problem, whose solution leads to a stepsize schedule that follows (a discretization of) a power-law distribution. Our proposed stepsize schedules and analysis extend to other methods, such as Optimistic Gradient (OG), and suggest broader applicability to general min-max optimization problems.

优化算法极小极大收敛加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。