用参考模型缓解联邦学习中的遗忘问题，提升稳定性和效率。
FedRef: Bayesian Fine-Tuning using a Reference Model to Mitigate Catastrophic Forgetting for Heterogeneous Federated Learning
- 基于参考模型的贝叶斯微调，通过正则化稳定全局更新
- 在非独立同分布数据下实现更快收敛和更高准确率
- 全在服务器端操作，显著降低客户端计算负担
联邦学习（FL）可在保护数据隐私的前提下实现分布式协同训练。然而，数据与系统异质性常引发灾难性遗忘和模型更新漂移，导致预测性能下降及客户端计算开销增加。为此，我们提出FedRef，一种利用历史全局模型构建参考模型的贝叶斯微调方法。该方法引入基于最大后验估计（MAP）的正则化项，将全局模型更新校准至时间聚合的参考模型，有效缓解遗忘并提升更新稳定性。与现有方法不同，FedRef所有微调操作均在服务器端完成，降低客户端计算负载的同时保持高效的全局优化。在图像分类（FEMNIST、CINIC-10）和医学图像分割（FeTS2022）任务上的实验表明，FedRef在异构非独立同分布（non-IID）设置下实现更优预测性能与更快收敛速度，同时显著降低客户端计算量，展现出对真实世界联邦学习场景的高效性与鲁棒性。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, data and system heterogeneity often cause catastrophic forgetting and unbounded drift in model updates, leading to degraded predictive performance and increased client-side computation. To address these challenges, we propose FedRef, a Bayesian fine-tuning method that leverages a reference model constructed from previous global models. FedRef integrates a MAP-based regularization term that calibrates global model updates toward a temporally aggregated reference model, thereby mitigating catastrophic forgetting and improving update stability. Unlike prior approaches, FedRef performs all fine-tuning operations on the server side, reducing client-side computational overhead while maintaining effective global optimization. Experiments on image classification (FEMNIST, CINIC-10) and medical image segmentation (FeTS2022) demonstrate that FedRef achieves superior predictive performance and faster convergence under heterogeneous, non-IID settings, while significantly lowering client-side computation compared with existing methods. These results highlight FedRef as an efficient and robust optimization framework for real-world FL scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。