arXiv:2411.01904cs.LGcs.CV2024-11被引 8

提出无重放的联邦持续学习框架,缓解遗忘与非独立同分布问题。

FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework

  • 用原型增强提示词实现无重放的轻量级协同学习
  • 在多种非独立同分布设置下保持稳定性能,避免灾难性遗忘
  • 适合资源受限且数据异构的联邦学习场景

联邦持续学习(FCL)旨在去中心化环境下从连续数据流中学习,同时缓解经典持续学习中的灾难性遗忘问题。现有方法通常采用典型重放机制,可能导致隐私泄露或带来额外存储与计算负担。本文提出一种高效且对非独立同分布(non-IID)鲁棒的联邦持续学习框架——联邦原型增强提示学习(FPPL)。FPPL可在不使用重放的前提下,通过原型增强的轻量提示词进行协作学习。客户端利用融合函数充分挖掘任务特定提示词中的知识,缓解遗忘;服务器端聚合全局原型,通过对比学习获得统一表征,降低非独立同分布导致的数据异质性影响。同时,本地上传的原型用于对分类器进行去偏,进一步缓解非独立同分布与遗忘带来的性能下降。实验表明,FPPL在高效设计下展现出显著性能,对多种非独立同分布程度均具有鲁棒性。代码已开源:https://github.com/ycheoo/FPPL。

原文摘要 · Abstract (English)

Federated continual learning (FCL) aims to learn from sequential data stream in the decentralized federated learning setting, while simultaneously mitigating the catastrophic forgetting issue in classical continual learning. Existing FCL methods usually employ typical rehearsal mechanisms, which could result in privacy violations or additional onerous storage and computational burdens. In this work, an efficient and non-IID robust federated continual learning framework, called Federated Prototype-Augmented Prompt Learning (FPPL), is proposed. The FPPL can collaboratively learn lightweight prompts augmented by prototypes without rehearsal. On the client side, a fusion function is employed to fully leverage the knowledge contained in task-specific prompts for alleviating catastrophic forgetting. Additionally, global prototypes aggregated from the server are used to obtain unified representation through contrastive learning, mitigating the impact of non-IID-derived data heterogeneity. On the server side, locally uploaded prototypes are utilized to perform debiasing on the classifier, further alleviating the performance degradation caused by both non-IID and catastrophic forgetting. Empirical evaluations demonstrate the effectiveness of FPPL, achieving notable performance with an efficient design while remaining robust to diverse non-IID degrees. Code is available at: https://github.com/ycheoo/FPPL.

联邦学习持续学习非独立同分布提示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。