解决边缘设备数据异构与计算瓶颈，提升联邦学习精度与效率
Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout
- 动态调整各设备模型权重以应对数据异构
- 自适应丢弃机制使准确率高15%，训练速度提升2倍
- 适合资源受限场景下的高效联邦学习应用
联邦学习（FL）是一种分布式机器学习方法，允许多个边缘设备协同训练全局模型。然而，设备间数据高度异构，非独立同分布（non-IID）数据会导致显著的准确率下降。同时，边缘设备计算和通信能力有限，易出现延迟设备，影响模型收敛速度。本文提出FedDHAD框架，包含两项新方法：动态异构模型聚合（FedDH）与自适应丢弃（FedAD）。FedDH根据各设备数据异构程度动态调整本地模型在聚合中的权重，缓解统计异构问题；FedAD针对异构设备执行神经元级自适应操作，在提升准确率的同时实现卓越效率。二者结合使FedDHAD在准确率上比现有最优方案最高提升6.7%，训练速度最快达2.02倍加速，计算成本降低最多15.0%。
原文摘要 · Abstract (English)
Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices. The data distributed among the edge devices is highly heterogeneous. Thus, FL faces the challenge of data distribution and heterogeneity, where non-Independent and Identically Distributed (non-IID) data across edge devices may yield in significant accuracy drop. Furthermore, the limited computation and communication capabilities of edge devices increase the likelihood of stragglers, thus leading to slow model convergence. In this paper, we propose the FedDHAD FL framework, which comes with two novel methods: Dynamic Heterogeneous model aggregation (FedDH) and Adaptive Dropout (FedAD). FedDH dynamically adjusts the weights of each local model within the model aggregation process based on the non-IID degree of heterogeneous data to deal with the statistical data heterogeneity. FedAD performs neuron-adaptive operations in response to heterogeneous devices to improve accuracy while achieving superb efficiency. The combination of these two methods makes FedDHAD significantly outperform state-of-the-art solutions in terms of accuracy (up to 6.7% higher), efficiency (up to 2.02 times faster), and computation cost (up to 15.0% smaller).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。