arXiv:2512.05327cs.LGmath.OC2025-12被引 1

提出兼顾通信与计算成本的非凸联邦优化新算法,提升效率。

Non-Convex Federated Optimization under Cost-Aware Client Selection

  • 设计带成本感知的客户端选择模型,明确区分不同策略开销。
  • 基于RG-SAGA梯度估计器,实现当前最优通信与局部计算复杂度。
  • 适合关注联邦学习实际部署效率的研究者与工程师。

不同的联邦优化算法通常采用不同的客户端选择策略:一些方法在每轮仅随机采样部分客户端通信,另一些则需定期与所有客户端通信,或采用混合方案。然而,现有优化方法的比较指标通常未区分这些策略,而它们在实践中往往带来不同的通信成本。为弥合这一差距,本文引入一个简单且自然的联邦优化模型,量化通信与本地计算复杂度。该模型可涵盖多种常见客户端选择策略,并为每种策略明确关联相应成本。在此框架下,提出一种新算法,在非凸优化场景下达到现有方法的最佳通信与本地计算复杂度。该算法基于不精确复合梯度法,结合精心设计的梯度估计器及迭代中的辅助子问题求解过程。梯度估计器基于SAGA(一种流行的方差缩减梯度估计器),我们首次推导出其新的方差上界,证明SAGA可利用函数相似性。进一步提出递归梯度技术,作为改进任意条件无偏梯度估计器误差界的一般方法,适用于SAGA和SVRG等。将该技术应用于SAGA,得到新型估计器RG-SAGA,其误差界优于原版。

原文摘要 · Abstract (English)

Different federated optimization algorithms typically employ distinct client-selection strategies: some methods communicate only with a randomly sampled subset of clients at each round, while others need to periodically communicate with all clients or use a hybrid scheme that combines both strategies. However, existing metrics for comparing optimization methods typically do not distinguish between these strategies, which often incur different communication costs in practice. To address this disparity, we introduce a simple and natural model of federated optimization that quantifies communication and local computation complexities. This new model allows for several commonly used client-selection strategies and explicitly associates each with a distinct cost. Within this setting, we propose a new algorithm that achieves the best-known communication and local complexities among existing federated optimization methods for non-convex optimization. This algorithm is based on the inexact composite gradient method with a carefully constructed gradient estimator and a special procedure for solving the auxiliary subproblem at each iteration. The gradient estimator is based on SAGA, a popular variance-reduced gradient estimator. We first derive a new variance bound for it, showing that SAGA can exploit functional similarity. We then introduce the Recursive-Gradient technique as a general way to potentially improve the error bound of a given conditionally unbiased gradient estimator, including both SAGA and SVRG. By applying this technique to SAGA, we obtain a new estimator, RG-SAGA, which has an improved error bound compared to the original one.

联邦学习非凸优化梯度估计通信效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。