提出自适应动量加速的非凸非光滑优化方法,提升收敛速度与稳定性。
An Inertial Block Proximal Linearized Method with Adaptive Momentum for Nonconvex and Nonsmooth Optimization

- 采用双阶段自适应动量策略更新外推参数,增强算法灵活性。
- 在稀疏非负矩阵分解等任务中,收敛速度优于现有先进方法。
- 适合求解带ℓ₀约束的非凸非光滑机器学习问题,如稀疏分解。
本文研究一类多块非凸非光滑优化问题,涵盖地震前异常分析与机器学习等应用。提出惯性块近端线性化方法结合双阶段自适应动量(IBPL⁺-TP),具有三大优势:(1) 引入双阶段自适应动量策略高效更新外推参数;(2) 允许使用两个不同外推点加速收敛;(3) 两外推点参数相互独立且无约束。在保持上述优点的同时,证明了目标函数单调下降、序列全局收敛至临界点,并建立了收敛速率。应用于稀疏非负矩阵分解(ℓ₀约束)和稀疏非负CP分解(ℓ₀约束)两类问题,数值实验表明该方法显著优于多个先进方法。
原文摘要 · Abstract (English)
In this paper, we consider a class of multiblock nonconvex nonsmooth optimization problems, which covers many applications such as the analysis of pre-earthquake anomalies and machine learning. To solve this class of problems, we propose the inertial block proximal linearized method with two-phase adaptive momentum (IBPL$^+$-TP). Compared to the current methods, our method possesses three main advantages: (1) it introduces a two-phase adaptive momentum strategy to effectively update the extrapolation parameters, (2) it allows using two different extrapolation points to accelerate the convergence, (3) it allows the extrapolation parameters of these two extrapolation points to be independent of and unconstrained by all other parameters. While maintaining the above advantages, we prove that our method ensures the monotonic convergence of the objective function of this class of problems, and we also prove that the sequence generated by our method globally converges to a critical point, as well as establish the convergence rate of our method. To demonstrate the effectiveness of our method, we apply it to solve two nonconvex and nonsmooth machine learning problems, namely sparse nonnegative matrix factorization with $\ell_0$-constraints and sparse nonnegative CP decomposition with $\ell_0$-constraints. The numerical experimental results on solving these problems show that our method outperforms several state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。