解决异步联邦学习中过时更新问题，无需数据即可提升模型性能。
Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
- 用无数据知识蒸馏复活过时客户端更新，仅在服务器端操作。
- 实验显示训练速度提升38.4%，最终准确率最高提高16.5%。
- 适合大规模异构设备上的高效联邦学习场景，尤其资源受限环境。
联邦学习（FL）可在不共享原始数据的前提下实现分布式客户端协作训练，但其可扩展性受限于同步开销。异步联邦学习（AFL）通过允许客户端独立通信缓解该问题，从而在大规模、硬件异构环境中提升实际运行效率。然而，异步性引入了基于过时全局模型计算的更新（即“过时”），可能破坏优化稳定性并阻碍收敛。本文提出FedRevive框架，通过无数据知识蒸馏（DFKD）复活过时更新。该方法将参数空间聚合与轻量级服务器端DFKD过程结合，无需数据即可将过时客户端更新的知识迁移至当前全局模型。采用元学习生成器合成伪样本，支持多教师蒸馏。混合聚合策略融合原始与DFKD更新，在缓解过时问题的同时保持AFL的可扩展性。在多个视觉与文本基准上实验表明，相比异步基线，FedRevive可实现最高达38.4%的训练加速和最高16.5%的最终准确率提升。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous federated learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, hardware-heterogeneous environments. However, asynchrony introduces updates computed on outdated global models (staleness) that can destabilize optimization and hinder convergence. We propose FedRevive, an AFL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a lightweight, server-side DFKD process that transfers knowledge from stale client updates to the current global model without access to data. A meta-learned generator synthesizes pseudo-samples used for multi-teacher distillation. A hybrid aggregation scheme that combines raw with DFKD updates effectively mitigates staleness while retaining AFL scalability. Experiments on various vision and text benchmarks show that FedRevive achieves faster training by up to 38.4% and higher final accuracy by up to 16.5% than asynchronous baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。