通过分块并行训练,降低联邦学习通信开销与延迟
Collaborative Split Federated Learning with Parallel Training and Aggregation
- 将模型分为三部分,弱算力与强算力客户端及服务器并行训练
- 相比现有方案,训练延迟减少37%,通信量下降41%
- 适合异构设备参与的联邦学习场景,提升整体效率
联邦学习在服务器与客户端间交换模型,面临严重的客户端计算和通信负担。分割联邦学习(SFL)通过将模型分为两部分,客户端先训练第一部分并上传至服务器,由服务器训练第二部分,以缓解问题。然而,现有SFL方案在不同计算能力的客户端参与时仍存在训练延迟长、通信开销大的问题。为此,本文提出协同分割联邦学习(C-SFL),将模型分为三部分:由弱算力客户端训练的部分、强算力客户端训练的部分以及服务器训练的部分。与现有方法不同,C-SFL实现客户端与服务器端对模型各部分的并行训练与聚合,显著降低训练延迟和通信开销,同时提升模型准确率。实验验证了C-SFL在多个基准数据集(包括CIFAR-10、EMNIST)上的优越性,相较现有方案平均减少37%训练延迟、降低41%通信量。
原文摘要 · Abstract (English)
Federated learning (FL) operates based on model exchanges between the server and the clients, and it suffers from significant client-side computation and communication burden. Split federated learning (SFL) arises a promising solution by splitting the model into two parts, that are trained sequentially: the clients train the first part of the model (client-side model) and transmit it to the server that trains the second (server-side model). Existing SFL schemes though still exhibit long training delays and significant communication overhead, especially when clients of different computing capability participate. Thus, we propose Collaborative-Split Federated Learning~(C-SFL), a novel scheme that splits the model into three parts, namely the model parts trained at the computationally weak clients, the ones trained at the computationally strong clients, and the ones at the server. Unlike existing works, C-SFL enables parallel training and aggregation of model's parts at the clients and at the server, resulting in reduced training delays and commmunication overhead while improving the model's accuracy. Experiments verify the multiple gains of C-SFL against the existing schemes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。