提出FENS框架，在单轮通信下逼近联邦学习精度
Revisiting Ensembling in One-Shot Federated Learning
- 分两阶段：先单轮本地训练上传，再用轻量聚合器协同训练
- 在CIFAR-10上比现有单次联邦学习高26.9%，仅比标准联邦学习低3.1%
- 通信量仅为标准联邦学习的十分之一，适合资源受限场景
联邦学习（FL）可在不共享原始数据的情况下训练模型，但标准方法需多轮通信，成本高。单次联邦学习（OFL）通过单轮通信降低开销，但准确率下降，尤其在数据异构时。本文提出FENS，一种新型联邦集成方法，兼具FL的高精度与OFL的低通信开销。其分两阶段：第一阶段客户端本地训练并上传模型，类似OFL；第二阶段客户端通过联邦学习协作训练一个轻量级预测聚合器。在多个数据集和异构水平下的实验验证了其有效性。在异构分布的CIFAR-10上，FENS准确率比当前最优OFL高出26.9%，仅比FL低3.1%。同时，其通信量最多为OFL的4.3倍，而FL至少是FENS的10.9倍。
原文摘要 · Abstract (English)
Federated learning (FL) is an appealing approach to training machine learning models without sharing raw data. However, standard FL algorithms are iterative and thus induce a significant communication cost. One-shot federated learning (OFL) trades the iterative exchange of models between clients and the server with a single round of communication, thereby saving substantially on communication costs. Not surprisingly, OFL exhibits a performance gap in terms of accuracy with respect to FL, especially under high data heterogeneity. We introduce FENS, a novel federated ensembling scheme that approaches the accuracy of FL with the communication efficiency of OFL. Learning in FENS proceeds in two phases: first, clients train models locally and send them to the server, similar to OFL; second, clients collaboratively train a lightweight prediction aggregator model using FL. We showcase the effectiveness of FENS through exhaustive experiments spanning several datasets and heterogeneity levels. In the particular case of heterogeneously distributed CIFAR-10 dataset, FENS achieves up to a 26.9% higher accuracy over state-of-the-art (SOTA) OFL, being only 3.1% lower than FL. At the same time, FENS incurs at most 4.3x more communication than OFL, whereas FL is at least 10.9x more communication-intensive than FENS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。