arXiv:2503.11828cs.LGcs.DC2025-03被引 2

分析去中心化联邦学习的收敛与泛化,揭示网络拓扑和数据非独立同分布的影响。

Performance Analysis of Decentralized Federated Learning Deployments

  • 比较线性、环形、星型、网格等网络拓扑对模型训练的影响。
  • 发现数据越非独立同分布,收敛速度越慢,且在真实数据集上验证了这一点。
  • 为实际部署去中心化联邦学习提供可参考的设计准则,适合隐私保护场景研究者。

智能手机和智能可穿戴设备的普及推动了中心化联邦学习(CFL)的发展,使其能在保护数据隐私的同时训练高性能机器学习模型。然而,CFL过度依赖中央服务器,导致延迟高、系统鲁棒性差。去中心化联邦学习(DFL)通过设备间直接协作,无需中央服务器,提升了系统灵活性。本文研究影响DFL模型收敛与泛化能力的关键因素,包括网络拓扑结构、数据非独立同分布(non-IID)程度及训练策略。首先推导了不同部署策略下的收敛速率;随后系统评估了线性、环形、星型、网格等网络拓扑在不同non-IID水平下的表现,并在经典模型、深度神经网络及大语言模型(LLMs)上,使用真实数据集进行测试。结果表明:当数据为独立同分布(IID)时,模型可收敛至最优;但non-IID程度越高,收敛速度越慢。研究结论为实际应用中的有效DFL部署提供了重要指导。

原文摘要 · Abstract (English)

The widespread adoption of smartphones and smart wearable devices has led to the widespread use of Centralized Federated Learning (CFL) for training powerful machine learning models while preserving data privacy. However, CFL faces limitations due to its overreliance on a central server, which impacts latency and system robustness. Decentralized Federated Learning (DFL) is introduced to address these challenges. It facilitates direct collaboration among participating devices without relying on a central server. Each device can independently connect with other devices and share model parameters. This work explores crucial factors influencing the convergence and generalization capacity of DFL models, emphasizing network topologies, non-IID data distribution, and training strategies. We first derive the convergence rate of different DFL model deployment strategies. Then, we comprehensively analyze various network topologies (e.g., linear, ring, star, and mesh) with different degrees of non-IID data and evaluate them over widely adopted machine learning models (e.g., classical, deep neural networks, and Large Language Models) and real-world datasets. The results reveal that models converge to the optimal one for IID data. However, the convergence rate is inversely proportional to the degree of non-IID data distribution. Our findings will serve as valuable guidelines for designing effective DFL model deployments in practical applications.

联邦学习去中心化非IID数据网络拓扑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。