用因果视角解决联邦学习中模型孤岛问题,实现低通信高性能
FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion
- 从因果角度发现数据异构导致模型拟合虚假关联
- 通过逐层融合其他客户端特征提升性能,通信成本为零
- 支持多客户端、异构训练,适合资源受限场景
单次联邦学习(OFL)通过仅聚合一次训练模型大幅降低通信开销,但现有方法性能远低于标准联邦学习。本文从因果视角揭示其性能下降源于‘模型孤岛’问题:由于数据异构,本地训练模型易拟合虚假相关性。我们观察到,通过引入其他客户端的中间特征可缓解该问题。基于此,提出新方法FuseFL,将神经网络分解为多个模块,采用自底向上的渐进式训练与融合策略进行特征增强,不增加额外通信开销。大量实验表明,FuseFL在性能上显著优于现有OFL与集成联邦学习方法,且具备高可扩展性、支持异构模型训练和低内存消耗。本工作首次利用因果分析来缓解OFL中的数据异构问题。
原文摘要 · Abstract (English)
One-shot Federated Learning (OFL) significantly reduces communication costs in FL by aggregating trained models only once. However, the performance of advanced OFL methods is far behind the normal FL. In this work, we provide a causal view to find that this performance drop of OFL methods comes from the isolation problem, which means that local isolatedly trained models in OFL may easily fit to spurious correlations due to the data heterogeneity. From the causal perspective, we observe that the spurious fitting can be alleviated by augmenting intermediate features from other clients. Built upon our observation, we propose a novel learning approach to endow OFL with superb performance and low communication and storage costs, termed as FuseFL. Specifically, FuseFL decomposes neural networks into several blocks, and progressively trains and fuses each block following a bottom-up manner for feature augmentation, introducing no additional communication costs. Comprehensive experiments demonstrate that FuseFL outperforms existing OFL and ensemble FL by a significant margin. We conduct comprehensive experiments to show that FuseFL supports high scalability of clients, heterogeneous model training, and low memory costs. Our work is the first attempt using causality to analyze and alleviate data heterogeneity of OFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。