arXiv:2601.09010math.OCcs.LG2026-01

提出高效解大规模优化问题的分解算法,提升实际应用中的收敛速度与稳定性。

Block Decomposable Methods for Large-Scale Optimization Problems

  • 设计自适应近似ADMM算法,允许子问题不精确求解,提升实用性。
  • 在动态误差下性能优于固定误差,收敛速度达同类方法最优水平。
  • 适用于非凸、凸及强凸函数,为随机块坐标下降提供理论保证。

本论文研究大规模优化问题的块可分解方法,重点聚焦交替方向乘子法(ADMM)和块坐标下降(BCD)方法。首先提出一种新型近似邻近ADMM算法,该方法对所有问题参数具有自适应性,且不精确求解近似增广拉格朗日子问题,显著提升了算法在各类实际问题中的效率。该不精确求解克服了传统ADMM在应用中的诸多挑战,所得算法在迭代次数上达到近似邻近ADMM类方法的最先进复杂度。其次,针对块邻近梯度方法引入了一种新的不精确邻近映射,并建立了其关键性质,推导出算法的收敛速率。在两种误差衰减条件下,算法收敛率与精确计算版本一致。数值实验表明,在动态误差环境下算法性能优于固定误差情形。最后,论文为随机块坐标下降法在霍尔德光滑函数类上的应用提供了收敛性保证,分别给出了非凸、凸和强凸函数的收敛速率,其结果与利普希茨光滑情况下的现有文献一致。

原文摘要 · Abstract (English)

This dissertation explores block decomposable methods for large-scale optimization problems. It focuses on alternating direction method of multipliers (ADMM) schemes and block coordinate descent (BCD) methods. Specifically, it introduces a new proximal ADMM algorithm and proposes two BCD methods. The first part of the research presents a new proximal ADMM algorithm. This method is adaptive to all problem parameters and solves the proximal augmented Lagrangian (AL) subproblem inexactly. This adaptiveness facilitates the highly efficient application of the algorithm to a broad swath of practical problems. The inexact solution of the proximal AL subproblem overcomes many key challenges in the practical applications of ADMM. The resultant algorithm obtains an approximate solution of an optimization problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM schemes. The second part of the research focuses on an inexact proximal mapping for the class of block proximal gradient methods. Key properties of this operator is established, facilitating the derivation of convergence rates for the proposed algorithm. Under two error decreases conditions, the algorithm matches the convergence rate of its exactly computed counterpart. Numerical results demonstrate the superior performance of the algorithm under a dynamic error regime over a fixed one. The dissertation concludes by providing convergence guarantees for the randomized BCD method applied to a broad class of functions, known as Hölder smooth functions. Convergence rates are derived for non-convex, convex, and strongly convex functions. These convergence rates match those furnished in the existing literature for the Lipschtiz smooth setting.

优化算法块分解收敛性分析近似求解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。