提出GQG统一框架，让优化算法更灵活地利用曲率信息。
Generalized Quadratic Gradient: A New Direction in Optimization via the Fusion of Positive-Definite Curvature Matrices and Gradients into A Unified Framework
- 将曲率矩阵与梯度融合，构建通用优化框架
- 支持任意正定曲率矩阵，不限于传统方法
- 适合研究高效优化算法的学者和工程师
二次梯度（QG）是一种牛顿型优化框架，通过在梯度更新中引入曲率信息，连接一阶梯度下降与二阶优化。简化二次梯度（SQG）在保持优化能力的同时降低复杂度，准二次梯度（QQG）则将二次梯度原理扩展至类似BFGS的拟牛顿方法。本文提出广义二次梯度（GQG），一个统一框架，将二次梯度原理推广到更广泛的牛顿型优化算法。通过抽象现有二次梯度方法的共同结构，我们发现二次梯度构造的根本要求并不要求特定的海森近似，如常数海森矩阵、对角海森近似或基于BFGS的海森代理。相反，只要满足局部二次模型的驻点条件，任何正定曲率矩阵均可使用。基于此视角，我们研究了使用多种非BFGS类正定海森代理构造广义二次梯度，为开发曲率感知优化算法提供了更广阔的基础。
原文摘要 · Abstract (English)
Quadratic Gradient (QG) is a Newton-type optimization framework that bridges first-order gradient descent and second-order optimization by incorporating curvature information into gradient updates. Simplified Quadratic Gradient (SQG) reduces the complexity of QG construction while preserving its optimization capability, whereas Quasi-Quadratic Gradient (QQG) extends the quadratic gradient principle to quasi-Newton methods such as BFGS. In this paper, we propose **Generalized Quadratic Gradient (GQG)**, a unified framework that extends the quadratic gradient principle to a broader class of Newton-type optimization algorithms. By abstracting the common structure of existing quadratic gradient methods, we show that the fundamental requirement of quadratic gradient construction is not limited to specific Hessian approximations, such as constant Hessian matrices, diagonal Hessian approximations, or BFGS-based Hessian surrogates. Instead, it can be generalized to any positive-definite curvature matrix satisfying the stationary condition of a local quadratic model. Based on this perspective, we investigate the construction of generalized quadratic gradients using various positive-definite Hessian surrogates beyond BFGS, providing a broader foundation for developing curvature-aware optimization algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。