通过双权重稀疏打包,提升资源受限下的联邦学习性能。
Tackling Resource-Constrained and Data-Heterogeneity in Federated Learning with Double-Weight Sparse Pack
- 客户端用余弦相似度筛选关键参数打包上传,降低通信开销。
- 服务器端引入方向与分布权重,提升稀疏更新的聚合效率和模型鲁棒性。
- 在4个数据集上优于10种主流方法,兼顾效率与精度,适合边缘设备部署。
联邦学习虽受广泛关注,但边缘客户端间的数据异质性仍严重制约模型性能。现有方法通过模型拆分与知识蒸馏缓解异质性,却忽视客户端有限的通信带宽与计算能力,难以平衡异质性处理与资源约束。为此,我们提出基于余弦稀疏参数打包与双权重聚合的个性化联邦学习方法FedCSPACK,有效利用客户端资源并减轻数据异质性影响。客户端根据余弦相似度打包并选择最具贡献的参数包上传,显著降低带宽需求;随后生成锚定于共享参数包的掩码矩阵,提升稀疏更新在服务器端的对齐与聚合效率。同时,在掩码中嵌入方向与分布距离权重,实现加权引导聚合,增强全局模型的鲁棒性与泛化能力。在四个数据集上使用十种先进方法进行的大量实验表明,FedCSPACK在保持高模型精度的同时,显著提升通信与计算效率。
原文摘要 · Abstract (English)
Federated learning has drawn widespread interest from researchers, yet the data heterogeneity across edge clients remains a key challenge, often degrading model performance. Existing methods enhance model compatibility with data heterogeneity by splitting models and knowledge distillation. However, they neglect the insufficient communication bandwidth and computing power on the client, failing to strike an effective balance between addressing data heterogeneity and accommodating limited client resources. To tackle this limitation, we propose a personalized federated learning method based on cosine sparsification parameter packing and dual-weighted aggregation (FedCSPACK), which effectively leverages the limited client resources and reduces the impact of data heterogeneity on model performance. In FedCSPACK, the client packages model parameters and selects the most contributing parameter packages for sharing based on cosine similarity, effectively reducing bandwidth requirements. The client then generates a mask matrix anchored to the shared parameter package to improve the alignment and aggregation efficiency of sparse updates on the server. Furthermore, directional and distribution distance weights are embedded in the mask to implement a weighted-guided aggregation mechanism, enhancing the robustness and generalization performance of the global model. Extensive experiments across four datasets using ten state-of-the-art methods demonstrate that FedCSPACK effectively improves communication and computational efficiency while maintaining high model accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。