将优化算法的收敛性理论扩展到非欧几何,揭示其在深层学习中的普适优势。
Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimization
- 在任意范数定义的球体上迭代优化,突破传统欧氏空间限制。
- 证明非欧版本仍具线性收敛与有限步终止特性,关键性质多数保留。
- 为理解现代优化算法(如Muon、Scion)提供理论蓝图,适合研究者参考。
最近提出的布罗克西马尔点方法(BPM)[Gruntkowska et al., 2025] 提供了一个基于在当前迭代点为中心的范数球上最小化目标函数的理想化优化框架。该方法对适当、闭合且凸函数具有出色的全局收敛性,可实现线性收敛并有限步内终止。然而,其理论分析此前仅限于欧几里得几何。与此同时,深度学习优化中涌现出的新趋势,如 Muon [Jordan et al., 2024] 和 Scion [Pethick et al., 2025] 等算法,表明使用非欧范数定义的球体进行最小化能更好匹配损失曲面的内在几何结构,展现出实际优势。本文探讨是否可将 BPM 的收敛理论推广至更一般的非欧几何设定。我们给出了肯定回答:大多数原方法的优雅保证在任意范数几何下依然成立。过程中厘清了哪些性质得以保持,哪些必然失效。我们的分析将非欧 BPM 定位为理解一大类几何感知优化算法的概念蓝图,揭示其实践有效性的底层原理。
原文摘要 · Abstract (English)
The recently proposed Broximal Point Method (BPM) [Gruntkowska et al., 2025] offers an idealized optimization framework based on iteratively minimizing the objective function over norm balls centered at the current iterate. It enjoys striking global convergence guarantees, converging linearly and in a finite number of steps for proper, closed and convex functions. However, its theoretical analysis has so far been confined to the Euclidean geometry. At the same time, emerging trends in deep learning optimization, exemplified by algorithms such as Muon [Jordan et al., 2024] and Scion [Pethick et al., 2025], demonstrate the practical advantages of minimizing over balls defined via non-Euclidean norms which better align with the underlying geometry of the associated loss landscapes. In this note, we ask whether the convergence theory of BPM can be extended to this more general, non-Euclidean setting. We give a positive answer, showing that most of the elegant guarantees of the original method carry over to arbitrary norm geometries. Along the way, we clarify which properties are preserved and which necessarily break down when leaving the Euclidean realm. Our analysis positions Non-Euclidean BPM as a conceptual blueprint for understanding a broad class of geometry-aware optimization algorithms, shedding light on the principles behind their practical effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。