arXiv:2603.04422cs.LGcs.AI2026-03

用指数移动平均与逻辑值蒸馏,提升联邦学习的鲁棒性与通信效率。

FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning

  • 服务器用指数移动平均平滑全局模型,结合客户端上传的预测逻辑值进行集成蒸馏。
  • 在非独立同分布数据下,准确率最高提升6%,通信轮次减少35%,上传量降至0.46MB以内。
  • 对恶意客户端有强抵抗力,适合部署于隐私敏感或设备异构的场景。

联邦学习在客户端数据异构且存在恶意行为时,常出现客户端漂移、收敛慢和通信开销大等问题。本文提出FedEMA-Distill,一种服务器端方法:结合全局模型的指数移动平均(EMA)与基于小规模公共代理数据集的客户端预测逻辑值集成蒸馏。客户端仅需标准本地训练并上传压缩后的逻辑值,支持不同模型架构,无需修改客户端代码。在CIFAR-10、CIFAR-100、FEMNIST和AG News上,使用Dirichlet-0.1标签偏斜设置,实验显示其准确率最高提升6%(CIFAR-100),达到目标准确率所需通信轮次减少30%-35%,每轮上行传输量降至0.09-0.46MB,约为全模型权重传输的十分之一。在服务器采用逐坐标中位数或截尾均值聚合逻辑值,可有效抵御最多20%的拜占庭客户端攻击,且在攻击下仍能生成校准良好的预测。结果表明,时间平滑与仅逻辑值聚合的结合,提供了一种通信高效、抗攻击且易于部署的联邦学习方案,兼容安全聚合与差分隐私,因仅交换聚合或混淆后的模型输出。

原文摘要 · Abstract (English)

Federated learning (FL) often degrades when clients hold heterogeneous non-Independent and Identically Distributed (non-IID) data and when some clients behave adversarially, leading to client drift, slow convergence, and high communication overhead. This paper proposes FedEMA-Distill, a server-side procedure that combines an exponential moving average (EMA) of the global model with ensemble knowledge distillation from client-uploaded prediction logits evaluated on a small public proxy dataset. Clients run standard local training, upload only compressed logits, and may use different model architectures, so no changes are required to client-side software while still supporting model heterogeneity across devices. Experiments on CIFAR-10, CIFAR-100, FEMNIST, and AG News under Dirichlet-0.1 label skew show that FedEMA-Distill improves top-1 accuracy by several percentage points (up to +5% on CIFAR-10 and +6% on CIFAR-100) over representative baselines, reaches a given target accuracy in 30-35% fewer communication rounds, and reduces per-round client uplink payloads to 0.09-0.46 MB, i.e., roughly an order of magnitude less than transmitting full model weights. Using coordinate-wise median or trimmed-mean aggregation of logits at the server further stabilizes training in the presence of up to 10-20% Byzantine clients and yields well-calibrated predictions under attack. These results indicate that coupling temporal smoothing with logits-only aggregation provides a communication-efficient and attack-resilient FL pipeline that is deployment-friendly and compatible with secure aggregation and differential privacy, since only aggregated or obfuscated model outputs are exchanged.

联邦学习知识蒸馏通信效率鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。