新算法实现非凸优化全局与局部最优收敛，无需调参。
A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees
- 用梯度历史构造正则项，结合共轭梯度与负曲率检测求解牛顿方程。
- 全局复杂度达 O(ε^{-3/2}) 次二阶查询，局部收敛速度为二次。
- 自适应不需已知海森矩阵利普希茨常数，适合物理信息神经网络训练。
寻找具有利普希茨连续海森矩阵的非凸函数的 ε-驻点是优化中的核心问题。正则化牛顿方法虽经典且研究广泛，但仍在全局与局部收敛之间存在权衡。是否能设计一种无需参数调整的算法，同时达到最优全局复杂度和二次局部收敛，仍是开放问题。为此，我们提出一类基于当前与前一迭代梯度构造的新正则项，并采用共轭梯度法结合负曲率监测求解正则化牛顿方程。该算法自适应，无需事先知道海森矩阵的利普希茨常数，全局复杂度分别为 O(ε^{-3/2}) 次二阶预言机调用及 ilde{O}(ε^{-7/4}) 次海森向量乘积。当迭代收敛至海森矩阵正定的点时，方法呈现二次局部收敛性。初步数值实验（包括物理信息神经网络训练）表明其性能具有竞争力。
原文摘要 · Abstract (English)
Finding an $ε$-stationary point of a nonconvex function with a Lipschitz continuous Hessian is a central problem in optimization. Regularized Newton methods are a classical tool and have been studied extensively, yet they still face a trade-off between global and local convergence. Whether a parameter-free algorithm of this type can simultaneously achieve optimal global complexity and quadratic local convergence remains an open question. To bridge this long-standing gap, we propose a new class of regularizers constructed from the current and previous gradients, and leverage the conjugate gradient approach with a negative curvature monitor to solve the regularized Newton equation. The proposed algorithm is adaptive, requiring no prior knowledge of the Hessian Lipschitz constant, and achieves a global complexity of $O(ε^{-3/2})$ in terms of the second-order oracle calls, and $\tilde{O}(ε^{-7/4})$ for Hessian-vector products, respectively. When the iterates converge to a point where the Hessian is positive definite, the method exhibits quadratic local convergence. Preliminary numerical results, including training the physics-informed neural networks, illustrate the competitiveness of our algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。