arXiv:2605.02838math.OCcs.AI2026-05

提出无需投影的二阶优化方法,加速在正交约束下的求解。

A second-order method landing on the Stiefel manifold via Newton$\unicode{x2013}$Schulz iteration

  • 通过牛顿-舒尔茨迭代构造法向分量,结合修正牛顿方程处理切向分量。
  • 在正交Procrustes、主成分分析等任务中收敛速度显著快于现有方法。
  • 适合高精度需求的正交矩阵优化问题,尤其适用于无投影场景。

无回缩方法为流形上的优化提供了低成本替代方案,但通常为一阶,难以满足高精度要求。为此,我们提出一种不依赖回缩操作的二阶优化方法,证明其具有局部二次(或不精确版本的超线性)收敛性。更新由两部分构成:(i) 沿约束函数等值面切向的分量,用于减小目标函数;(ii) 沿同一等值面法向的分量,用于消除不可行性。其中法向分量通过牛顿-舒尔茨迭代构建,该迭代是正交化的一种不动点算法。我们进一步揭示了牛顿-舒尔茨迭代与Stiefel流形间的几何联系——其沿法空间移动。切向分量则通过引入牛顿-舒尔茨的修正牛顿方程进行建模。在正交Procrustes问题、主成分分析及真实数据独立成分分析上的数值实验表明,所提方法优于现有方法。

原文摘要 · Abstract (English)

Retraction-free approaches offer attractive low-cost alternatives to Riemannian methods on the Stiefel manifold, but they are often first-order, which may limit the efficiency under high-accuracy requirements. To this end, we propose a second-order method landing on the Stiefel manifold without invoking retractions, which is proved to enjoy local quadratic (or superlinear for its inexact variant) convergence. The update consists of the sum of (i) a component tangent to the level set of the constraint-defining function that aims to reduce the objective and (ii) a component normal to the same level set that reduces the infeasibility. Specifically, we construct the normal component via Newton$\unicode{x2013}$Schulz, a fixed-point iteration for orthogonalization. Moreover, we establish a geometric connection between the Newton$\unicode{x2013}$Schulz iteration and Stiefel manifolds, in which Newton$\unicode{x2013}$Schulz moves along the normal space. For the tangent component, we formulate a modified Newton equation that incorporates Newton$\unicode{x2013}$Schulz. Numerical experiments on the orthogonal Procrustes problem, principal component analysis, and real-data independent component analysis illustrate that the proposed method performs better than the existing methods.

优化流形二阶方法正交约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。