arXiv:2605.08488math.OCcs.LG2026-05被引 1

用控制理论统一分析加速优化算法的稳定性,可直接验证收敛性。

A Unified Lyapunov-IQC Framework for Uniform Stability of Smooth Quadratic First-Order Accelerated Optimizers

论文配图:A Unified Lyapunov-IQC Framework for Uniform Stability of Smooth Quadratic First-Order Accelerated Optimizers
图 1 · 摘自论文原文
  • 将优化算法建模为反馈系统,结合李雅普诺夫函数与积分二次约束
  • 通过半定规划求解稳定性条件,给出光滑二次问题下的统一稳定界
  • 方法适用于NAG等复杂算法,适合研究优化理论与鲁棒控制交叉领域

本文提出统一的李雅普诺夫-积分二次约束(Lyapunov-IQC)框架,用于分析一阶加速优化算法在β-光滑与γ-强凸条件下的统一稳定性。经典分析依赖随机采样下的逐项控制,难以推广至含动量的加速方法。本文首先借助李雅普诺夫函数扩展该方法,获得光滑二次NAG的统一稳定性界,并辅以小规模实验验证。进一步将一阶加速优化器建模为线性系统与梯度算子构成的Lur'e型反馈连接,利用扇区积分二次约束编码β-光滑性和γ-强凸性。在此框架下,统一稳定性可通过求解有限维线性矩阵不等式(LMI)的可行性问题进行认证,该问题可由半定规划(SDP)求解。以NAG为例,验证了经典稳定性界可由此框架恢复。结果揭示了优化动力学与鲁棒控制理论之间的深层结构关联,提供了一种模块化、可复现的数值认证方法,适用于日益复杂的优化算法,基于凸优化工具实现可靠稳定性与泛化行为分析。

原文摘要 · Abstract (English)

We develop a unified Lyapunov-integral quadratic constraint (IQC) framework for establishing uniform stability of first-order accelerated optimization algorithms in the $β$-smooth and $γ$-strongly convex regime. Classical analyses of uniform stability, such as the work of Hardt, Recht, and Singer for stochastic gradient descent (SGD), rely on direct coupling arguments and case-by-case control of iterate differences under random sampling. Extending such arguments to accelerated methods, such as Nesterov Accelerated Gradient (NAG), is complicated by the presence of higher-order state dynamics induced by momentum. We first extend this classical approach with the use of Lyapunov functions to provide a uniform stability bound for smooth quadratic NAG, and supplement this result with small-scale numerical experiments. We then extend this framework by modeling first-order accelerated optimizers as Lur'e-type feedback interconnections between a linear dynamical system and a (non-linear) gradient operator. $β$-Smoothness and $γ$-strong convexity are encoded a sector IQC inequality. Under this representation, uniform stability is certified via the existence of a quadratic Lyapunov function satisfying a finite-dimensional linear matrix inequality (LMI) in the form of a feasibility problem, which can be solved via semi-definite programming (SDP). We instantiate this framework for NAG and show how classical uniform stability bounds can be recovered via this framework. These results underscore a structural connection between optimization dynamics and robust control theory, providing a modular methodology for reliable and reproducible numerical certification of uniform stability and generalization behavior of first-order methods via convex optimization tools that is adaptable to increasingly complex optimization algorithms.

优化算法稳定性分析控制理论半定规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。