提出无需投影的优化算法，收敛更快且计算更省。
Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints
- 不需投影直接更新，利用局部Riemannian PŁ条件保证收敛
- 在非凸光滑函数上实现线性收敛，实测性能媲美顶尖方法
- 适合大规模矩阵优化，尤其适用于主成分分析与深度学习
许多经典和现代机器学习算法需要在正交约束下求解优化问题。传统可行方法需在梯度下降后执行Stiefel流形上的回缩操作，计算成本较高。近期提出的无回缩方法——着陆算法（landing algorithm）作为高效替代方案。本文针对主成分分析和深度神经网络训练中常见的正交约束问题，研究该算法并基于局部Riemannian PŁ条件，首次建立其对光滑非凸函数的线性收敛率。数值实验表明，着陆算法性能与当前最先进的回缩方法相当，但计算开销显著降低。
原文摘要 · Abstract (English)
Many classical and modern machine learning algorithms require solving optimization tasks under orthogonality constraints. Solving these tasks with feasible methods requires a gradient descent update followed by a retraction operation on the Stiefel manifold, which can be computationally expensive. Recently, an infeasible retraction-free approach, termed the landing algorithm, was proposed as an efficient alternative. Motivated by the common occurrence of orthogonality constraints in tasks such as principle component analysis and training of deep neural networks, this paper studies the landing algorithm and establishes a novel linear convergence rate for smooth non-convex functions using only a local Riemannian PŁ condition. Numerical experiments demonstrate that the landing algorithm performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。