通过只交换部分信息坐标,实现高效低通信的去中心化联邦学习。
Decentralized Federated Learning by Partial Message Exchange
- 仅在相邻节点间随机交换稀疏坐标,大幅降低通信开销。
- 在梯度局部Lipschitz和双随机通信矩阵条件下线性收敛。
- 适合大规模异构数据场景,兼顾隐私、精度与通信效率。
去中心化联邦学习(DFL)作为一种无服务器范式,已在大规模异构网络中实现协同学习。然而,其仍面临数据异构性、理论分析限制性假设以及标准通信或隐私增强技术导致收敛性能下降等根本挑战。为此,本文提出新型算法PaME(基于部分消息交换的DFL)。核心思想是仅允许随机选择的稀疏坐标在相邻节点间交换。该方法显著降低通信成本,同时保持高隐私性且不牺牲准确性。基于严格分析,算法在梯度局部Lipschitz连续、通信矩阵双随机的温和假设下实现线性收敛。这两个条件较现有方法更为宽松,有效应对数据异构性。大量数值实验表明,其性能优于多个代表性去中心化学习算法。
原文摘要 · Abstract (English)
Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks. However, it continues to face fundamental challenges, including data heterogeneity, restrictive assumptions for theoretical analysis, and degraded convergence when standard communication- or privacyenhancing techniques are applied. To overcome these drawbacks, this paper develops a novel algorithm, PaME (DFL by Partial Message Exchange). The central principle is to allow only randomly selected sparse coordinates to be exchanged between two neighbor nodes. Consequently, PaME achieves substantial reductions in communication costs while still preserving a high level of privacy, without sacrificing accuracy. Moreover, grounded in rigorous analysis, the algorithm is shown to converge at a linear rate under the gradient to be locally Lipschitz continuous and the communication matrix to be doubly stochastic. These two mild assumptions not only dispense with many restrictive conditions commonly imposed by existing DFL methods but also enables PaME to effectively address data heterogeneity. Furthermore, comprehensive numerical experiments demonstrate its superior performance compared with several representative decentralized learning algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。