arXiv:2502.02002math.OCcs.LG2025-02被引 6

新优化算法在非光滑非凸问题上实现线性收敛,突破传统方法瓶颈。

The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications

  • 用球约束替代二次惩罚,设计新型近端算子提升优化稳定性。
  • 在非光滑凸情形下实现线性收敛且有限步内完成,远优于经典方法。
  • 理论框架可启发自适应步长、加速算法等实际应用设计。

非光滑与非凸全局优化在诸多应用中面临重大挑战,传统基于梯度的方法常表现不佳。本文提出球近端点法(Ball-Proximal Point Method, BPM),受经典近端点法(PPM)启发,通过将二次距离惩罚替换为球约束,引入球近端算子。令人意外的是,在非光滑凸情形下,BPM展现出线性收敛性,并可在有限步内完成收敛,显著优于传统PPM的亚线性速率。进一步引入球凸性概念,证明BPM在更弱假设下仍保持全局收敛性,适用于更广泛的潜在非凸优化问题。正如PPM曾推动梯度下降、自适应步长、加速方法等发展,BPM亦可作为理论蓝图,激发后续高效算法的设计。

原文摘要 · Abstract (English)

Non-smooth and non-convex global optimization poses significant challenges across various applications, where standard gradient-based methods often struggle. We propose the Ball-Proximal Point Method, Broximal Point Method, or Ball Point Method (BPM) for short - a novel algorithmic framework inspired by the classical Proximal Point Method (PPM) (Rockafellar, 1976), which, as we show, sheds new light on several foundational optimization paradigms and phenomena, including non-convex and non-smooth optimization, acceleration, smoothing, adaptive stepsize selection, and trust-region methods. At the core of BPM lies the ball-proximal ("broximal") operator, which arises from the classical proximal operator by replacing the quadratic distance penalty by a ball constraint. Surprisingly, and in sharp contrast with the sublinear rate of PPM in the nonsmooth convex regime, we prove that BPM converges linearly and in a finite number of steps in the same regime. Furthermore, by introducing the concept of ball-convexity, we prove that BPM retains the same global convergence guarantees under weaker assumptions, making it a powerful tool for a broader class of potentially non-convex optimization problems. Just like PPM plays the role of a conceptual method inspiring the development of practically efficient algorithms and algorithmic elements, e.g., gradient descent, adaptive step sizes, acceleration (Ahn & Sra, 2020), and "W" in AdamW (Zhuang et al., 2022), we believe that BPM should be understood in the same manner: as a blueprint and inspiration for further development.

优化算法非凸优化收敛性分析近端方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。