arXiv:2508.09685cs.LGcs.IT2025-08

不加正则项的梯度下降也能线性收敛,证明了其隐式正则化特性。

Global Convergence Analysis of Vanilla Gradient Descent for Asymmetric Matrix Completion

  • 用谱初始化的原始梯度下降法,通过留一法证明线性收敛。
  • 实验显示算法计算成本更低,完成精度与现有方法相当。
  • 揭示了梯度下降在无显式正则时仍具隐式正则能力,适合理论研究者。

本文研究非对称低秩矩阵补全问题,将其建模为无约束非凸优化问题,目标函数为非线性最小二乘,采用梯度下降求解。以往方法通常引入正则项以保证收敛,但数值实验和梯度流分析表明,移除正则项并不影响收敛性能。通过引入留一法技术,我们归纳证明:使用谱初始化的原始梯度下降在高概率下实现线性收敛率。此外,我们发现平衡正则项在迭代过程中保持小范数,揭示了梯度下降的隐式正则性质。实验结果表明,该算法计算开销更低,同时保持与其它梯度下降算法相当的补全性能。

原文摘要 · Abstract (English)

This paper investigates the asymmetric low-rank matrix completion problem, which can be formulated as an unconstrained non-convex optimization problem with a nonlinear least-squares objective function, and is solved via gradient descent methods. Previous gradient descent approaches typically incorporate regularization terms into the objective function to guarantee convergence. However, numerical experiments and theoretical analysis of the gradient flow both demonstrate that the elimination of regularization terms in gradient descent algorithms does not adversely affect convergence performance. By introducing the leave-one-out technique, we inductively prove that the vanilla gradient descent with spectral initialization achieves a linear convergence rate with high probability. Besides, we demonstrate that the balancing regularization term exhibits a small norm during iterations, which reveals the implicit regularization property of gradient descent. Empirical results show that our algorithm has a lower computational cost while maintaining comparable completion performance compared to other gradient descent algorithms.

矩阵补全梯度下降非凸优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。