提出C-FLAG方法，缓解联邦学习中持续学习的灾难性遗忘问题。
On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
- 基于梯度记忆与聚合，在边缘端更新模型参数。
- 理论证明收敛速度达O(1/√T)，有效减少遗忘与偏差。
- 适用于需要持续学习的隐私保护场景，如物联网设备。
机器学习的终极目标是实现持续联邦学习（CFL），以提升AI系统在流式数据下的效率、隐私保护和可扩展性。CFL的核心挑战是克服全局灾难性遗忘——新任务训练导致旧任务准确率下降。本文提出一种基于重放记忆的联邦学习策略C-FLAG，结合边缘端对记忆数据的梯度更新与当前数据的聚合梯度。我们提供了C-FLAG的收敛性分析，证明其在T轮通信下以O(1/√T)的速率收敛，同时缓解遗忘与偏差。通过构建最小化灾难性遗忘的优化子问题，将CFL转化为具有自适应学习率的迭代算法，实现跨任务无缝学习。实验表明，C-FLAG在任务增量与类别增量设置下均优于多个先进基线方法，显著提升准确率并降低遗忘率。
原文摘要 · Abstract (English)
The holy grail of machine learning is to enable Continual Federated Learning (CFL) to enhance the efficiency, privacy, and scalability of AI systems while learning from streaming data. The primary challenge of a CFL system is to overcome global catastrophic forgetting, wherein the accuracy of the global model trained on new tasks declines on the old tasks. In this work, we propose Continual Federated Learning with Aggregated Gradients (C-FLAG), a novel replay-memory based federated strategy consisting of edge-based gradient updates on memory and aggregated gradients on the current data. We provide convergence analysis of the C-FLAG approach which addresses forgetting and bias while converging at a rate of $O(1/\sqrt{T})$ over $T$ communication rounds. We formulate an optimization sub-problem that minimizes catastrophic forgetting, translating CFL into an iterative algorithm with adaptive learning rates that ensure seamless learning across tasks. We empirically show that C-FLAG outperforms several state-of-the-art baselines on both task and class-incremental settings with respect to metrics such as accuracy and forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。