解决联邦学习中不同客户端知识遗忘问题,用投影记忆替代生成回放。
When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learning

- 用服务器端投影记忆替代生成输入回放,避免跨模态迁移困难。
- 在异构任务流下仍保持高知识保留率,优于传统方法。
- 适合资源受限的联邦学习场景,尤其适用于多模态数据。
联邦类增量学习(FCIL)在客户端观察到不同标签子集、任务进度不一且对同一语义概念监督不均时变得尤为困难。现有方法常通过输入空间合成来保留旧知识,但在异构任务流下易失效且难以跨模态迁移。为此,我们提出PRO框架,将合成输入回放替换为投影回放编排。为避免外部预训练影响,所有方法均在相同热启动条件下评估。PRO在服务器端维护紧凑的类别级投影记忆,客户端则基于当前样本与旧投影记忆进行平衡的伪多任务训练。为进一步缓解表示漂移,引入PRO-MAX,通过邻域加权记忆对齐增强性能,同时保持服务器仅聚合模型更新和记忆统计的轻量原则。在图像、文本和图基准上,PRO与PRO-MAX在异构流下均显著提升保留率与最终性能,且在同质FCIL中保持竞争力。即使基线方法拥有更大回放预算,仍因监督不平衡与阶段错位而退化,表明回放数量无法解决回放质量问题。弱任务诊断进一步显示,更大的回放不匹配与下游性能下降正相关,而我们的方法能更好维持投影记忆与动态表征的一致性。
原文摘要 · Abstract (English)
Federated class-incremental learning (FCIL) becomes substantially harder when clients observe different label subsets, progress through tasks at different stages, and provide uneven supervision for the same semantic concepts. Existing FCIL methods often preserve old knowledge through input-space synthesis, but they can be fragile under heterogeneous task streams and difficult to transfer across modalities. To alleviate such issues, we propose PRO, a framework that replaces synthetic input replay with projected rehearsal orchestration. To remove external pretraining, we evaluate all methods under the same warmup. After this, PRO maintains compact class-level projected memories on the server and allows clients perform balanced pseudo multi-task training over current examples and old projected memories. To handle stronger representation drift, we further introduce PRO-MAX, which augments PRO with neighborhood-weighted memory alignment while preserving the same server-light principle that the server only aggregates model updates and memory statistics. Across image, text, and graph benchmarks, PRO and PRO-MAX improve retention and final utility under heterogeneous streams while remaining competitive in homogeneous FCIL. Even when baselines are given expanded replay budgets, they degrade under supervision imbalance and stage misalignment, indicating that replay quantity alone does not resolve replay-quality failures. Additional weak-task diagnostics further show that larger replay mismatch is associated with larger downstream degradation, while our method keeps projected memories better aligned with the evolving representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。