提出轻量级联邦集成蒸馏方法，显著提升异构客户端下的模型性能。
Provably Near-Optimal Federated Ensemble Distillation with Negligible Overhead
- 基于客户端判别器与服务器生成器的协同训练，设计可证明近最优的加权策略。
- 在多个图像分类任务上超越基线，且通信开销、隐私泄露和计算负担可忽略。
- 适用于有无预存服务器数据的场景，适合实际部署的联邦学习系统。
联邦集成蒸馏通过利用客户端预测生成无标签服务器数据的伪标签，并基于该伪标签数据训练服务器模型，以缓解客户端异构性问题。该方法中的伪标签生成质量高度依赖于客户端预测的加权方式，尤其在高度异构环境下更为关键。受生成对抗网络理论启发，我们提出一种可证明近最优的加权机制，该机制通过在服务器端分发生成器并结合本地数据训练客户端判别器实现。在多种图像分类任务上的实验表明，该方法显著优于现有基线。进一步分析显示，该方法带来的额外通信开销、客户端隐私泄露及计算负担在有无预存服务器数据的场景下均可以忽略不计。
原文摘要 · Abstract (English)
Federated ensemble distillation addresses client heterogeneity by generating pseudo-labels for an unlabeled server dataset based on client predictions and training the server model using the pseudo-labeled dataset. The unlabeled server dataset can either be pre-existing or generated through a data-free approach. The effectiveness of this approach critically depends on the method of assigning weights to client predictions when creating pseudo-labels, especially in highly heterogeneous settings. Inspired by theoretical results from GANs, we propose a provably near-optimal weighting method that leverages client discriminators trained with a server-distributed generator and local datasets. Our experiments on various image classification tasks demonstrate that the proposed method significantly outperforms baselines. Furthermore, we show that the additional communication cost, client-side privacy leakage, and client-side computational overhead introduced by our method are negligible, both in scenarios with and without a pre-existing server dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。