arXiv:2504.11903cs.LGcs.DC2025-04被引 4

FedCanon降低联邦学习中非凸问题的计算开销,提升异构数据下的模型性能。

FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data

  • 将局部更新与近端操作解耦,每轮只需服务器执行一次近端计算。
  • 在异构数据下准确率优于现有方法,收敛速度达到亚线性甚至线性。
  • 适合处理非凸目标函数和非光滑正则项的分布式优化场景。

组合式联邦学习为带正则项的机器学习问题提供通用框架。然而,现有方法常面临客户端需执行高成本近端操作、且性能受数据异构影响的问题。为此,我们提出新算法 FedCanon,用于求解可能非凸的损失函数与弱凸、可能非光滑正则项组成的优化问题。通过将近端映射与局部更新解耦,FedCanon 每轮迭代仅需服务器执行一次近端评估,显著降低整体计算成本;同时引入控制变量缓解异构数据导致的客户端漂移。整个架构避免了原始-对偶交替的复杂子问题。理论分析首次为该跳过近端操作框架在一般非凸设置下提供了严格收敛保证:在无有界异构性假设条件下,可实现亚线性收敛率,在 Polyak-Łojasiewicz 条件下可达线性收敛率。大量实验表明,相比当前最优方法,FedCanon 在准确率和计算效率上均表现更优,尤其在异构数据分布下优势明显。

原文摘要 · Abstract (English)

Composite federated learning offers a general framework for solving machine learning problems with additional regularization terms. However, existing methods often face significant limitations: many require clients to perform computationally expensive proximal operations, and their performance is frequently vulnerable to data heterogeneity. To overcome these challenges, we propose a novel composite federated learning algorithm called \textbf{FedCanon}, designed to solve the optimization problems comprising a possibly non-convex loss function and a weakly convex, potentially non-smooth regularization term. By decoupling proximal mappings from local updates, FedCanon requires only a single proximal evaluation on the server per iteration, thereby reducing the overall proximal computation cost. Concurrently, it integrates control variables into local updates to mitigate the client drift arising from data heterogeneity. The entire architecture avoids the complex subproblems of primal-dual alternatives. The theoretical analysis provides the first rigorous convergence guarantees for this proximal-skipping framework in the general non-convex setting. It establishes that FedCanon achieves a sublinear convergence rate, and a linear rate under the Polyak-Łojasiewicz condition, without the restrictive bounded heterogeneity assumption. Extensive experiments demonstrate that FedCanon outperforms the state-of-the-art methods in terms of both accuracy and computational efficiency, particularly under heterogeneous data distributions.

联邦学习非凸优化异构数据近端计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。