arXiv:2412.05823cs.LGcs.AI2024-12NeurIPS被引 13

针对边缘设备异构性,提出模型融合剪枝与自适应正则化,提升跨域联邦学习性能。

DapperFL: Domain Adaptive Federated Learning with Model Fusion Pruning for Edge Devices

  • 通过融合多域知识剪枝本地模型,生成个性化紧凑模型。
  • 在多域数据上提升准确率最高达2.28%,模型体积压缩20%~80%。
  • 适合资源受限的边缘设备部署,尤其适用于数据分布不一致场景。

联邦学习(FL)在边缘计算环境中成为主流机器学习范式,使边缘设备能在不共享私有数据的前提下协作优化全局模型。然而,现有FL框架因边缘计算固有的系统异构性,尤其在本地数据存在领域偏移时,性能显著下降。本文提出异构联邦学习框架DapperFL,以提升多领域下的模型表现。DapperFL引入专用的模型融合剪枝(MFP)模块,通过融合本地与剩余领域的知识,生成个性化的紧凑本地模型,有效应对系统异构挑战。同时设计领域自适应正则化(DAR)模块,利用剪枝后模型生成的正则项,学习跨域鲁棒表示。此外,提出一种适配异构架构与权重的聚合算法,实现多样化本地模型的有效集成。我们在真实世界异构客户端平台上实现DapperFL,实验结果表明,在多个基准数据集上的多域测试中,其性能优于多种先进FL框架,最高提升2.28%,且模型体积压缩率达20%至80%。代码已开源:https://github.com/jyzgh/DapperFL。

原文摘要 · Abstract (English)

Federated learning (FL) has emerged as a prominent machine learning paradigm in edge computing environments, enabling edge devices to collaboratively optimize a global model without sharing their private data. However, existing FL frameworks suffer from efficacy deterioration due to the system heterogeneity inherent in edge computing, especially in the presence of domain shifts across local data. In this paper, we propose a heterogeneous FL framework DapperFL, to enhance model performance across multiple domains. In DapperFL, we introduce a dedicated Model Fusion Pruning (MFP) module to produce personalized compact local models for clients to address the system heterogeneity challenges. The MFP module prunes local models with fused knowledge obtained from both local and remaining domains, ensuring robustness to domain shifts. Additionally, we design a Domain Adaptive Regularization (DAR) module to further improve the overall performance of DapperFL. The DAR module employs regularization generated by the pruned model, aiming to learn robust representations across domains. Furthermore, we introduce a specific aggregation algorithm for aggregating heterogeneous local models with tailored architectures and weights. We implement DapperFL on a realworld FL platform with heterogeneous clients. Experimental results on benchmark datasets with multiple domains demonstrate that DapperFL outperforms several state-of-the-art FL frameworks by up to 2.28%, while significantly achieving model volume reductions ranging from 20% to 80%. Our code is available at: https://github.com/jyzgh/DapperFL.

联邦学习边缘计算模型剪枝域适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。