针对边缘设备资源差异,提出高效联邦学习框架,减少通信开销。
FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices
- 资源充足客户端用深度分类模型,受限设备用轻量生成模型。
- 两阶段服务器处理:生成数据并融合知识,提升全局模型性能。
- 解决知识遗忘问题,无需额外数据集,适合边缘计算场景。
联邦学习(FL)在边缘计算中日益普及,大量异构客户端在资源受限或充足的条件下运行。传统FL的迭代训练带来显著计算与通信开销,对资源受限的边缘设备不友好。一次性联邦学习(One-shot FL)有效降低通信开销,而模型异构型联邦学习则应对客户端间计算资源差异。然而,现有方法在管理模型异构的一次性联邦学习时仍面临挑战,常导致全局模型性能不佳或依赖辅助数据集。为此,我们提出新型联邦学习框架FedMHO,利用资源充足的客户端上的深度分类模型,以及资源受限设备上的轻量生成模型。服务器端采用两阶段过程:数据生成与知识融合。此外,引入FedMHO-MD和FedMHO-SD以缓解知识融合阶段的知识遗忘问题,并设计无监督数据优化方案提升合成样本质量。大量实验表明,该方法在多种设置下优于现有最先进基线。
原文摘要 · Abstract (English)
Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional FL introduces significant computation and communication overhead, which is unfriendly for resource-constrained edge devices. One-shot FL has emerged as a promising approach to mitigate communication overhead, and model-heterogeneous FL solves the problem of diverse computing resources across clients. However, existing methods face challenges in effectively managing model-heterogeneous one-shot FL, often leading to unsatisfactory global model performance or reliance on auxiliary datasets. To address these challenges, we propose a novel FL framework named FedMHO, which leverages deep classification models on resource-sufficient clients and lightweight generative models on resource-constrained devices. On the server side, FedMHO involves a two-stage process that includes data generation and knowledge fusion. Furthermore, we introduce FedMHO-MD and FedMHO-SD to mitigate the knowledge-forgetting problem during the knowledge fusion stage, and an unsupervised data optimization solution to improve the quality of synthetic samples. Comprehensive experiments demonstrate the effectiveness of our methods, as they outperform state-of-the-art baselines in various experimental setups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。