提出Stiefel流形上精确闭式更新的优化方法，提升正交约束优化效率。
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
- 基于Stiefel流形设计矩阵正交约束的精确闭式更新
- 新算法Skewon在非凸平滑条件下实现一阶收敛
- 相比迭代近似法更高效，适合正交约束优化任务
我们研究了近期提出的矩阵感知优化方法Muon在Stiefel流形上的应用。该流形由列正交矩阵构成，在机器学习与科学计算中广泛应用。现有Muon在该流形上的扩展依赖于启发式、近似或迭代更新，计算效率各异。本文证明对应的Stiefel Muon更新存在精确闭式解，并据此提出Skewon算法，用于正交约束优化，具有高效实现。进一步建立了Skewon在光滑非凸设定下的首阶收敛保证。
原文摘要 · Abstract (English)
We study Muon, a recently proposed matrix-aware optimization method, in the context of the Stiefel manifold. This manifold consists of matrices with orthonormal columns and is ubiquitous in machine learning and scientific computing. Existing extensions of Muon to this manifold rely on heuristic, approximate, or iterative updates with varying computational efficiency. We show that the corresponding Stiefel Muon update admits an exact closed-form solution and use this result to develop Skewon, a practical algorithm for orthogonality-constrained optimization with an efficient implementation. We further establish first-order convergence guarantees for Skewon in the smooth non-convex setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。