改进联邦学习通信效率，支持灵活步长并分析收敛性。
Analysis of regularized federated learning
- 提出可调步长的无循环局部梯度下降算法，降低通信开销。
- 在非凸场景下，基于Polyak-Łojasiewicz条件给出收敛速率。
- 揭示强凸情况下的收敛必要充分条件，适合研究联邦学习者。
联邦学习是处理异构大数据和隐私保护的有效机器学习方法。带正则化的联邦学习方法可通过控制通信概率水平来调节中心与本地设备间的通信频率。通常采用随机梯度下降在异构数据上实现该类方法以降低通信成本。本文研究一种名为无循环局部梯度下降（Loopless Local Gradient Descent）的算法，其优势在于通过控制概率降低期望通信量。我们通过允许灵活步长对方法进行改进，并在非凸设置下开展新的收敛性分析，同时保留标准强凸情形。在非凸情形中，当光滑目标函数满足Polyak-Łojasiewicz条件时，导出了收敛速率。在强凸情形中，给出了期望收敛的充分必要条件。
原文摘要 · Abstract (English)
Federated learning is an efficient machine learning tool for dealing with heterogeneous big data and privacy protection. Federated learning methods with regularization can control the level of communications between the central and local machines. Stochastic gradient descent is often used for implementing such methods on heterogeneous big data, to reduce the communication costs. In this paper, we consider such an algorithm called Loopless Local Gradient Descent which has advantages in reducing the expected communications by controlling a probability level. We improve the method by allowing flexible step sizes and carry out novel analysis for the convergence of the algorithm in a non-convex setting in addition to the standard strongly convex setting. In the non-convex setting, we derive rates of convergence when the smooth objective function satisfies a Polyak-Łojasiewicz condition. When the objective function is strongly convex, a sufficient and necessary condition for the convergence in expectation is presented.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。