提出新框架，让一阶优化算法在非光滑条件下也能高效避开鞍点。
Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
- 基于自界正则性构建统一分析框架
- 首次实现非光滑下二阶驻点的收敛保证
- 适用于机器学习中常见非光滑场景
我们研究在不满足传统光滑性（梯度和/或海森矩阵Lipschitz）条件下的非凸函数优化问题，采用一阶方法。由于光滑性假设在机器学习中理论与实践中均过于严格，近期大量工作致力于在广义光滑性条件下用一阶方法寻找一阶驻点。本文提出一个新框架，可系统分析一大类一阶优化算法（称为下降过程）在广义光滑性下的收敛性。我们利用该框架分析算法在广义光滑性下收敛至一阶及二阶驻点的过程，并首次建立了在广义光滑性下一阶方法收敛到二阶驻点的保证。我们证明多个经典例子均符合本框架，并揭示其实际意义。
原文摘要 · Abstract (English)
We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods. Smoothness is a restrictive assumption in machine learning in both theory and practice, motivating significant recent work on finding first order stationary points of functions satisfying generalizations of smoothness with first order methods. We develop a novel framework that lets us systematically study the convergence of a large class of first-order optimization algorithms (which we call decrease procedures) under generalizations of smoothness. We instantiate our framework to analyze the convergence of first order optimization algorithms to first and \textit{second} order stationary points under generalizations of smoothness. As a consequence, we establish the first convergence guarantees for first order methods to second order stationary points under generalizations of smoothness. We demonstrate that several canonical examples fall under our framework, and highlight practical implications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。