提出首个无需投影的加速自适应优化算法，突破维度依赖瓶颈。
Optimal Projection-Free Adaptive SGD for Matrix Optimization
- 改进李昂算法稳定性分析，避免额外超参数调优。
- 首次实现无投影、带加速的自适应优化，收敛性更优。
- 适用于非凸非光滑问题，适合大规模矩阵优化场景。
近期，Jiang 等人 [2026] 提出 Leon，一种实用的单边 Shampoo [Xie et al., 2025a, An et al., 2025] 在线凸优化变体，无需每轮计算代价高昂的二次投影。然而，现有分析表明，Leon 需要调优其预条件器中的额外超参数，且在梯度有界假设之外无法获得维度无关的收敛保证。本文通过证明 Leon 预条件器的某些稳定性性质，解决了该问题：无需调优额外超参数，并进一步提出首个带 Nesterov 加速的无投影自适应 SGD 变体，无需每轮计算投影。作为附加贡献，我们在非光滑非凸设置下获得了改进的维度无关收敛率，并建立统一分析框架，支持 (块) 对角预条件器的加速无投影自适应随机梯度下降。
原文摘要 · Abstract (English)
Recently, Jiang et al. [2026] developed Leon, a practical variant of One-sided Shampoo [Xie et al., 2025a, An et al., 2025] algorithm for online convex optimization, which does not require computing a costly quadratic projection at each iteration. Unfortunately, according to the existing analysis, Leon requires tuning an additional hyperparameter in its preconditioner and cannot achieve dimension-independent convergence guarantees for convex optimization problems beyond the bounded gradients assumption. In this paper, we resolve this issue by proving certain stability properties of Leon's preconditioner. Using our improved analysis, we show that tuning the extra hyperparameter can be avoided and, more importantly, develop the first practical variant of One-sided Shampoo with Nesterov acceleration, which does not require computing projections at each iteration. As a side contribution, we obtain improved dimension-independent rates in the non-smooth non-convex setting and develop a unified analysis of the proposed algorithm, which yields accelerated projection-free adaptive SGD with (block-)diagonal preconditioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。