解决联邦学习中随时间变化的数据分布导致的遗忘问题。
FlashbackCL: Mitigating Temporal Forgetting in Federated Learning
- 用动态衰减的标签计数和设备感知重放缓冲区应对时间漂移。
- 在CIFAR-10上相对基准提升6.9%~10.0%,遗忘率降低最多68%。
- 适合处理数据分布随时间变化的边缘设备联邦学习场景。
联邦学习在基础模型与边缘模型部署中日益面临客户端数据分布随时间漂移的问题,而现有缓解遗忘的方法假设各客户端分布恒定。最强的近期方法Flashback使用逐类标签计数作为知识代理,但在时间分布漂移下会失准,使全局模型锚定在过时的类别平衡上。本文提出一种可即插即用的Flashback扩展——FlashbackCL,包含:(i) 时间衰减的标签计数;(ii) 基于类别平衡水库采样的设备感知重放缓冲区(CBRS);(iii) 在公共蒸馏集上进行服务器端主动核心集筛选。实验显示,在50个客户端、三种受控时间漂移模式下的CIFAR-10上,FlashbackCL相较Flashback实现6.9%至10.0%的相对性能提升,同时将时间遗忘减少高达68%。5种变体消融实验表明,CBRS重放是关键组件。此外,其在静态CIFAR-100上也提升3.5分,表明类别平衡重放能同时缓解空间异质性与时间漂移。
原文摘要 · Abstract (English)
Federated Learning (FL) of foundation and edge models increasingly targets deployments where client data distributions drift over time, yet existing forgetting-mitigation methods assume each client's distribution is stationary. Flashback, the strongest recent FL method against cross-client (spatial) forgetting, uses monotonically accumulating per-class label counts as a knowledge proxy; this proxy becomes miscalibrated under temporal distribution shift and anchors the global model to an outdated class balance. We formalise temporal forgetting in FL with a per-phase metric isolated from protocol-level fluctuations and propose Flashback Continual Learning (FlashbackCL), a drop-in extension of Flashback with (i) temporally-decayed label counts; (ii) a device-aware replay buffer with Class-Balanced Reservoir Sampling (CBRS); and (iii) server-side active coreset curation on the public distillation set. The results show that FlashbackCL achieves 6.9% to 10.0% relative improvement relative to Flashback, on CIFAR-10 with 50 clients and three controlled temporal shift modes, while simultaneously reducing temporal forgetting by up to 68%. A 5-variant ablation identifies CBRS replay as the critical component. FlashbackCL also improves Flashback by 3.5 points on stationary CIFAR-100, suggesting that class-balanced replay regularises spatial heterogeneity as well as temporal shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。