arXiv:2410.18862cs.LG2024-10被引 8

提出一种去中心化个性化联邦学习方法,降低通信开销并提升异构数据下的模型性能。

FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning

  • 基于软聚类的个性化建模,允许客户端按数据分布选择性更新模型。
  • 在低连通网络中仍能收敛,测试准确率优于现有去中心化个性化算法。
  • 适合数据异构性强、通信受限的分布式场景,如边缘设备协同训练。

联邦学习近年来成为分布式客户端使用本地数据协作训练机器学习模型的流行框架。传统联邦学习依赖中心服务器进行模型聚合,而最新进展采用去中心化架构,使客户端间可直接交换模型,消除单点故障。然而,现有去中心化框架通常假设所有客户端训练共享模型。针对异构数据分布,个性化每个客户端的模型可显著提升性能。本文提出 FedSPD,一种面向去中心化设置的高效个性化联邦学习算法,并证明其在低连通网络中仍能学习高精度模型。为提供收敛性理论保障,引入基于聚类的框架,使不同数据簇间达成模型共识,同时在各客户端对簇的混合形式进行个性化。该灵活性允许根据数据分布选择性更新模型,相比以往去中心化个性化联邦学习工作,显著降低通信成本。在真实世界数据集上的实验结果表明,FedSPD 在低连通网络场景下优于多种去中心化个性化联邦学习变体。

原文摘要 · Abstract (English)

Federated learning has recently gained popularity as a framework for distributed clients to collaboratively train a machine learning model using local data. While traditional federated learning relies on a central server for model aggregation, recent advancements adopt a decentralized framework, enabling direct model exchange between clients and eliminating the single point of failure. However, existing decentralized frameworks often assume all clients train a shared model. Personalizing each client's model can enhance performance, especially with heterogeneous client data distributions. We propose FedSPD, an efficient personalized federated learning algorithm for the decentralized setting, and show that it learns accurate models even in low-connectivity networks. To provide theoretical guarantees on convergence, we introduce a clustering-based framework that enables consensus on models for distinct data clusters while personalizing to unique mixtures of these clusters at different clients. This flexibility, allowing selective model updates based on data distribution, substantially reduces communication costs compared to prior work on personalized federated learning in decentralized settings. Experimental results on real-world datasets show that FedSPD outperforms multiple decentralized variants of personalized federated learning algorithms, especially in scenarios with low-connectivity networks.

联邦学习去中心化个性化聚类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。