arXiv:2409.01251cs.LGcs.DC2024-09AAAI被引 7

解决联邦学习中设备异步上传导致模型偏差的问题。

GAS: Generative Activation-Aided Asynchronous Split Federated Learning

论文配图:GAS: Generative Activation-Aided Asynchronous Split Federated Learning
图 1 · 摘自论文原文
  • 服务器用缓冲区管理异步传来的激活值和模型。
  • 生成虚拟激活值缓解资源强设备带来的更新偏差。
  • 理论证明收敛性更优,适合设备差异大的场景。

分割式联邦学习(SFL)在客户端与服务器间分片协同训练共享模型,客户端向服务器传输激活值和本地模型以供更新。现有研究假设激活值与本地模型同步传输,但客户端计算与通信能力差异大,导致数据异步到达,显著降低性能。为此,本文提出异步SFL框架GAS,服务器端设置激活缓冲区与模型缓冲区分别管理异步传输的激活值与模型。由于资源丰富客户端频繁传输激活值,易造成服务器模型更新偏差,GAS通过基于接收激活值构建各标签的激活分布,并根据偏差程度生成合成激活值,辅助服务器模型更新,确保更准确的优化。我们推导出更紧的收敛界,实验验证了方法有效性。代码已开源。

原文摘要 · Abstract (English)

Split Federated Learning (SFL) splits and collaboratively trains a shared model between clients and server, where clients transmit activations and client-side models to server for updates. Recent SFL studies assume synchronous transmission of activations and client-side models from clients to server. However, due to significant variations in computational and communication capabilities among clients, activations and client-side models arrive at server asynchronously. The delay caused by asynchrony significantly degrades the performance of SFL. To address this issue, we consider an asynchronous SFL framework, where an activation buffer and a model buffer are embedded on the server to manage the asynchronously transmitted activations and client-side models, respectively. Furthermore, as asynchronous activation transmissions cause the buffer to frequently receive activations from resource-rich clients, leading to biased updates of the server-side model, we propose Generative activations-aided Asynchronous SFL (GAS). In GAS, the server maintains an activation distribution for each label based on received activations and generates activations from these distributions according to the degree of bias. These generative activations are then used to assist in updating the server-side model, ensuring more accurate updates. We derive a tighter convergence bound, and our experiments demonstrate the effectiveness of the proposed method. The code is available at https://github.com/eejiarong/GAS.

联邦学习异步通信生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。