arXiv:2504.14741cs.LGmath.OC2025-04

提出交替梯度下降与最小化方法,提升部分解耦优化问题的效率。

AltGDmin: Alternating GD and Minimization for Partly-Decoupled (Federated) Optimization

  • 交替使用梯度下降和最小化,针对变量分块中一个子集易求解的问题设计新算法。
  • 在低秩矩阵补全等任务中,收敛速度比传统方法快,且通信开销更低。
  • 适合联邦学习、压缩感知等场景,尤其适用于变量间存在解耦结构的问题。

本文提出一种新型优化框架AltGDmin,适用于多种交替最小化(AltMin)广泛应用的问题。该方法基于块坐标下降思想,将变量分为两组Za、Zb。当其中一组(如Zb)的最小化可快速求解且目标函数对Za可微时,该方法更高效。通常因问题在Zb上具有解耦结构,使得各子问题可快速求解,从而提升整体效率并实现联邦学习中的通信高效性。适用问题包括:低秩列压缩感知(LRCS)、低秩矩阵补全(LRMC)、鲁棒主成分分析(robust PCA)、相位恢复及其稀疏/低秩扩展、张量型问题(如张量LRCS)以及部分离散问题(如聚类、无标签传感、混合线性回归)。其中LRCS应用于多任务表示学习、少样本学习、联邦压缩与加速动态MRI;LRMC与鲁棒PCA则广泛用于推荐系统、计算机视觉与视频分析。

原文摘要 · Abstract (English)

This article describes a novel optimization solution framework, called alternating gradient descent (GD) and minimization (AltGDmin), that is useful for many problems for which alternating minimization (AltMin) is a popular solution. AltMin is a special case of the block coordinate descent algorithm that is useful for problems in which minimization w.r.t one subset of variables keeping the other fixed is closed form or otherwise reliably solved. Denote the two blocks/subsets of the optimization variables Z by Za, Zb, i.e., Z = {Za, Zb}. AltGDmin is often a faster solution than AltMin for any problem for which (i) the minimization over one set of variables, Zb, is much quicker than that over the other set, Za; and (ii) the cost function is differentiable w.r.t. Za. Often, the reason for one minimization to be quicker is that the problem is ``decoupled" for Zb and each of the decoupled problems is quick to solve. This decoupling is also what makes AltGDmin communication-efficient for federated settings. Important examples where this assumption holds include (a) low rank column-wise compressive sensing (LRCS), low rank matrix completion (LRMC), (b) their outlier-corrupted extensions such as robust PCA, robust LRCS and robust LRMC; (c) phase retrieval and its sparse and low-rank model based extensions; (d) tensor extensions of many of these problems such as tensor LRCS and tensor completion; and (e) many partly discrete problems where GD does not apply -- such as clustering, unlabeled sensing, and mixed linear regression. LRCS finds important applications in multi-task representation learning and few shot learning, federated sketching, and accelerated dynamic MRI. LRMC and robust PCA find important applications in recommender systems, computer vision and video analytics.

优化算法联邦学习低秩优化交替最小化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。