提出CSE-FSL,降低联邦分层学习的通信与存储开销
Federated Split Learning with Improved Communication and Storage Efficiency
- 用辅助网络本地更新客户端权重,服务器仅存一个模型
- 通过选择性传输压缩数据,通信量显著减少
- 理论保证非凸损失下的收敛性,适合资源受限设备
联邦学习(FL)虽是流行的分布式机器学习方案,但边缘设备面临高昂的通信与计算成本。联邦分层学习(FSL)通过拆分模型架构并行训练子模型,减轻了边缘设备的计算负担,但仍需在每轮全局通信中频繁传输压缩数据和梯度,导致高通信开销;且服务器需为每个客户端维护独立的局部模型,造成巨大存储压力。本文提出一种新型通信与存储高效联邦分层学习方法——CSE-FSL,利用辅助网络在客户端本地更新权重,服务器仅保留单一模型,避免服务器梯度频繁上传,大幅降低存储需求。此外,通过在选定轮次传输压缩数据,进一步减少客户端发送的数据量。我们对CSE-FSL提供了理论分析,严格证明其在非凸损失函数下的收敛性。大量实验结果表明,在真实世界联邦学习任务中,CSE-FSL相较于现有FSL方案实现了显著的通信效率提升。
原文摘要 · Abstract (English)
Federated learning (FL) is one of the popular distributed machine learning (ML) solutions but incurs significant communication and computation costs at edge devices. Federated split learning (FSL) can train sub-models in parallel and reduce the computational burden of edge devices by splitting the model architecture. However, it still requires a high communication overhead due to transmitting the smashed data and gradients between clients and the server in every global round. Furthermore, the server must maintain separate partial models for every client, leading to a significant storage requirement. To address these challenges, this paper proposes a novel communication and storage efficient federated split learning method, termed CSE-FSL, which utilizes an auxiliary network to locally update the weights of the clients while keeping a single model at the server, hence avoiding frequent transmissions of gradients from the server and greatly reducing the storage requirement of the server. Additionally, a new model update method of transmitting the smashed data in selected epochs can reduce the amount of smashed data sent from the clients. We provide a theoretical analysis of CSE-FSL, rigorously guaranteeing its convergence under non-convex loss functions. The extensive experimental results further indicate that CSE-FSL achieves a significant communication reduction over existing FSL solutions using real-world FL tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。