将联邦持续学习建模为控制问题,用虚拟队列抑制遗忘。
Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem

- 引入虚拟队列追踪跨任务遗忘累积,实现稳定与适应的平衡。
- 在多个数据异构场景下,准确率优于现有方法,遗忘率显著降低。
- 无需梯度投影或额外通信,参数可调,适合真实分布式系统。
联邦持续学习(FCL)是现实分布式学习系统的核心,要求模型在客户端面对顺序、非独立同分布的数据时持续适应,同时缓解灾难性遗忘和客户端漂移。现有方法将持续学习视为一系列本地优化问题,通过聚合耦合,依赖回放、正则化或基于投影的约束等启发式机制。然而,遗忘在FCL中是长期且分布式的现象,源于任务随时间演进与客户端异质性的交互,现有方法未显式调控。本文将FCL建模为随机控制问题,提出基于李雅普诺夫漂移-惩罚(DPP)优化的联邦队列调控持续学习(FedQCL)。FedQCL引入虚拟队列以追踪任务与客户端间的遗忘累积,从而显式控制稳定性-可塑性权衡。通过优化DPP目标,该方法在提升当前任务性能的同时,利用队列机制提供可解释、可调的平衡机制,仅需单一参数调节,无需梯度投影或额外通信开销。在标准基准测试(包括Split-CIFAR-10、Split-CIFAR-100、Split-TinyImageNet)上的实验表明,FedQCL在异构数据分布下优于现有先进基线,兼具更高准确率与更少遗忘。
原文摘要 · Abstract (English)
Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems, requiring models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift. Existing approaches formulate continual learning (CL) as a sequence of per-task optimization problems, applied locally at each client and coupled through aggregation, using heuristic mechanisms such as replay, regularization, or projection-based constraints. However, forgetting in FCL is inherently a long-term, distributed phenomenon, arising from the interaction of temporal task evolution and cross-client heterogeneity, which is not explicitly regulated. In this work, we cast FCL as a stochastic control problem and propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization. FedQCL introduces virtual queues to track the accumulation of forgetting across tasks and clients, enabling explicit control of the stability-plasticity trade-off. By optimizing a DPP objective, the method jointly improves current-task performance while the queue-based formulation provides an interpretable and tunable mechanism to balance adaptation and retention through a single parameter, without requiring gradient projection or additional communication overhead. Empirical evaluations on standard benchmarks, including Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet, demonstrate that FedQCL outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。