arXiv:2411.06042cs.LGcs.NI2024-11中稿 · publication in IEE…被引 1

针对无线网络中设备资源受限问题,提出个性化分层联邦学习方法。

Personalized Hierarchical Split Federated Learning in Wireless Networks

  • 将模型分为客户端和服务器端,仅在客户端训练部分参数,降低通信开销。
  • 冻结分类器并全局训练主干网络,再微调各客户端分类器实现个性化。
  • 实验表明该方法在保持高效通信的同时显著提升个性化性能,适合边缘计算场景。

无线网络中大规模机器学习受极端资源限制,分布式客户端面临电池与计算能力不足的问题。传统方案需大量信息交换,而分层联邦学习(SFL)通过将模型拆分至客户端和服务器端,仅在客户端训练部分模型,缓解了这一挑战。然而实际应用需要适配用户个人任务的个性化模型。为此,我们提出个性化分层联邦学习(PHSFL)算法:由于客户端数据分布差异虽大但特征属性相似,故仅训练模型主体部分,保持分类器随机初始化并冻结。通过理论分析模型分割与分层聚合对全局模型的影响后,先训练全局模型,再对每个客户端的分类器进行微调以获得个性化模型。实验表明,尽管未训练分类器的全局模型性能已接近现有方法,但微调后的个性化模型显著提升表现。

原文摘要 · Abstract (English)

Extreme resource constraints make large-scale machine learning (ML) with distributed clients challenging in wireless networks. On the one hand, large-scale ML requires massive information exchange between clients and server(s). On the other hand, these clients have limited battery and computation powers that are often dedicated to operational computations. Split federated learning (SFL) is emerging as a potential solution to mitigate these challenges, by splitting the ML model into client-side and server-side model blocks, where only the client-side block is trained on the client device. However, practical applications require personalized models that are suitable for the client's personal task. Motivated by this, we propose a personalized hierarchical split federated learning (PHSFL) algorithm that is specially designed to achieve better personalization performance. More specially, owing to the fact that regardless of the severity of the statistical data distributions across the clients, many of the features have similar attributes, we only train the body part of the federated learning (FL) model while keeping the (randomly initialized) classifier frozen during the training phase. We first perform extensive theoretical analysis to understand the impact of model splitting and hierarchical model aggregations on the global model. Once the global model is trained, we fine-tune each client classifier to obtain the personalized models. Our empirical findings suggest that while the globally trained model with the untrained classifier performs quite similarly to other existing solutions, the fine-tuned models show significantly improved personalized performance.

联邦学习边缘计算个性化模型分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。