统一分析自适应优化方法在非凸问题中的收敛性，涵盖多种主流算法。
A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muon
- 提出统一框架，用自适应预处理梯度优化非凸问题。
- 在合理假设下，证明所有方法全局收敛，无需小步长或有界梯度。
- 支持变量分组使用不同几何结构，适合复杂模型优化场景。
针对非凸无约束优化问题，提出一种统一的前阶优化算法框架，采用自适应预处理梯度，包含全量与对角线AdaGrad、AdaNorm以及自适应版Muon等主流方法。该框架允许不同变量组使用异构几何结构，同时保持统一的收敛性分析。在合理假设梯度噪声方差的前提下，对所有方法进行了完全随机化的全局收敛速率分析，无需假设梯度有界或步长足够小，适用于带两种动量机制的情况。
原文摘要 · Abstract (English)
A unified framework for first-order optimization algorithms fornonconvex unconstrained optimization is proposed that uses adaptivelypreconditioned gradients and includes popular methods such as full anddiagonal AdaGrad, AdaNorm, as well as an adpative variant of Muon. This framework also allows combining heterogeneous geometries across different groups of variables while preserving a unified convergence analysis. A fully stochastic global rate-of-convergence analysis is conducted for all methods in the framework, with andwithout two types of momentum, using reasonable assumptions on the variance of the gradient oracle and without assuming bounded stochastic gradients or small enough stepsize.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。