解决联邦学习中客户端数据不平衡问题,从源头提升模型性能。
Federated Balanced Learning
- 在客户端通过知识填充与采样实现样本平衡,不增加数据量。
- 提出知识对齐与丢弃策略,缓解生成数据与真实数据的差距。
- 支持异构客户端灵活部署,适用于复杂真实场景。
联邦学习通过共享模型参数而非数据实现协同训练,但在非独立同分布(non-iid)设置下,全局模型易出现客户端漂移,严重影响最终性能。现有方法多在模型已偏移后通过损失或梯度修正,忽视客户端样本分布的影响。本文重新思考客户端角色,提出联邦均衡学习(FBL),从源头解决该问题:在客户端固定数据量约束下,利用边缘侧生成模型进行知识填充与知识采样,实现样本平衡。同时设计知识对齐策略以弥合合成数据与真实数据的差距,并引入知识丢弃策略进行正则化。方法可扩展至真实复杂场景,支持不同客户端采用各异策略,进一步提升性能。大量实验表明,本方法优于当前最优基线。代码将在接受后公开。
原文摘要 · Abstract (English)
Federated learning is a paradigm of joint learning in which clients collaborate by sharing model parameters instead of data. However, in the non-iid setting, the global model experiences client drift, which can seriously affect the final performance of the model. Previous methods tend to correct the global model that has already deviated based on the loss function or gradient, overlooking the impact of the client samples. In this paper, we rethink the role of the client side and propose Federated Balanced Learning, i.e., FBL, to prevent this issue from the beginning through sample balance on the client side. Technically, FBL allows unbalanced data on the client side to achieve sample balance through knowledge filling and knowledge sampling using edge-side generation models, under the limitation of a fixed number of data samples on clients. Furthermore, we design a Knowledge Alignment Strategy to bridge the gap between synthetic and real data, and a Knowledge Drop Strategy to regularize our method. Meanwhile, we scale our method to real and complex scenarios, allowing different clients to adopt various methods, and extend our framework to further improve performance. Numerous experiments show that our method outperforms state-of-the-art baselines. The code is released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。