解决联邦学习中类别增量的样本不平衡问题,提升模型泛化能力。
Class-wise Balancing Data Replay for Federated Class-Incremental Learning
- 通过全局视角构建类级记忆,实现均衡采样。
- 动态调整分类温度,缓解多数类过拟合、少数类忽略问题。
- 在异构数据下显著提升准确率,适合实际联邦场景使用。
联邦类别增量学习(FCIL)旨在多个客户端间协作处理持续增长的任务。现有方法中的数据重放虽能缓解遗忘,但受限于类间不平衡——既包括重放缓冲区内的类分布不均,也涉及重放类与新类之间的差异。为此,我们提出一种类级平衡的数据重放方法(FedCBDR),通过全局协调机制构建类级内存,并重新加权学习目标以缓解上述不平衡。具体而言,FedCBDR包含两个关键模块:1)全局视角重放模块在保护隐私的前提下重构前序任务的全局表征,指导类感知且重要性敏感的采样策略,实现均衡重放;2)任务感知温度缩放模块根据任务动态,在类和实例层面自适应调整对数概率的温度,降低模型对多数类的过信心,增强对少数类的敏感性。实验表明,FedCBDR在异构数据分布下实现了类级别的均衡采样,在早期与近期任务不平衡条件下提升了泛化性能,相比六种先进方法,Top-1准确率提升2%-15%。
原文摘要 · Abstract (English)
Federated Class Incremental Learning (FCIL) aims to collaboratively process continuously increasing incoming tasks across multiple clients. Among various approaches, data replay has become a promising solution, which can alleviate forgetting by reintroducing representative samples from previous tasks. However, their performance is typically limited by class imbalance, both within the replay buffer due to limited global awareness and between replayed and newly arrived classes. To address this issue, we propose a class wise balancing data replay method for FCIL (FedCBDR), which employs a global coordination mechanism for class-level memory construction and reweights the learning objective to alleviate the aforementioned imbalances. Specifically, FedCBDR has two key components: 1) the global-perspective data replay module reconstructs global representations of prior task in a privacy-preserving manner, which then guides a class-aware and importance-sensitive sampling strategy to achieve balanced replay; 2) Subsequently, to handle class imbalance across tasks, the task aware temperature scaling module adaptively adjusts the temperature of logits at both class and instance levels based on task dynamics, which reduces the model's overconfidence in majority classes while enhancing its sensitivity to minority classes. Experimental results verified that FedCBDR achieves balanced class-wise sampling under heterogeneous data distributions and improves generalization under task imbalance between earlier and recent tasks, yielding a 2%-15% Top-1 accuracy improvement over six state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。