arXiv:2412.15536cs.DCcs.LG2024-12被引 11

选对分割层能让联邦学习更高效,尤其对共享模型版本效果显著

The Impact of Cut Layer Selection in Split Federated Learning

  • 按不同层分割神经网络,客户端和服务器分担计算
  • 共享模型版SFL-V2性能受分割层影响大,选对能超越传统方法
  • 适合关注联邦学习优化的工程师与研究者

分割联邦学习(SFL)是一种结合联邦学习与分割学习的分布式机器学习范式。在SFL中,神经网络在某一分割层处被切分,前端层部署于客户端,后端层由训练服务器持有。现有研究主要集中在算法开发,但对分割层选择如何影响模型性能的定量分析仍不充分。本文通过数值与理论分析,系统研究了分割层选择对SFL性能与收敛性的影响。结果表明,SFL-V1对分割层选择不敏感,与理论一致;而在四个数据集、两种神经网络上的实验显示,SFL-V2的性能显著依赖于分割层选择。恰当的分割层可使SFL-V2在非同质数据下优于FedAvg。

原文摘要 · Abstract (English)

Split Federated Learning (SFL) is a distributed machine learning paradigm that combines federated learning and split learning. In SFL, a neural network is partitioned at a cut layer, with the initial layers deployed on clients and remaining layers on a training server. There are two main variants of SFL: SFL-V1 where the training server maintains separate server-side models for each client, and SFL-V2 where the training server maintains a single shared model for all clients. While existing studies have focused on algorithm development for SFL, a comprehensive quantitative analysis of how the cut layer selection affects model performance remains unexplored. This paper addresses this gap by providing numerical and theoretical analysis of SFL performance and convergence relative to cut layer selection. We find that SFL-V1 is relatively invariant to the choice of cut layer, which is consistent with our theoretical results. Numerical experiments on four datasets and two neural networks show that the cut layer selection significantly affects the performance of SFL-V2. Moreover, SFL-V2 with an appropriate cut layer selection outperforms FedAvg on heterogeneous data.

联邦学习模型分割分布式训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。