arXiv:2501.05775cs.LGcs.DC2025-01被引 1

解决联邦学习中时空异构数据难题,提升模型适应性和泛化能力。

STHFL: Spatio-Temporal Heterogeneous Federated Learning

  • 设计全局-局部动态原型框架,动态适配客户端数据分布变化。
  • 在长尾数据下,全局原型补充小样本类别知识,提升训练效果。
  • 通过原型更新机制缓解灾难性遗忘,适合持续学习场景。

联邦学习是一种保护数据隐私并允许多设备协作训练模型的新范式。以往研究虽针对非独立同分布数据和域间异构问题提出多种方法,但忽略了域内随任务增加而产生的时空异构性。实际应用中,全局数据通常呈长尾分布而非均衡分布。为此,本文提出新型设定——时空异构联邦学习(STHFL)。特别地,设计了全局-局部动态原型(GLDP)框架:每个客户端模型包含可动态适应不同数据分布的个性化层;针对长尾分布,全局原型为样本少的类别提供互补知识,且不泄露隐私;随着任务增加,先前任务生成的本地原型指导当前任务训练,缓解灾难性遗忘;同时,客户端本地原型训练后,通过移动平均法更新全局-局部原型。实验表明,该方法在STHFL场景下显著优于现有先进方法。

原文摘要 · Abstract (English)

Federated learning is a new framework that protects data privacy and allows multiple devices to cooperate in training machine learning models. Previous studies have proposed multiple approaches to eliminate the challenges posed by non-iid data and inter-domain heterogeneity issues. However, they ignore the \textbf{spatio-temporal} heterogeneity formed by different data distributions of increasing task data in the intra-domain. Moreover, the global data is generally a long-tailed distribution rather than assuming the global data is balanced in practical applications. To tackle the \textbf{spatio-temporal} dilemma, we propose a novel setting named \textbf{Spatio-Temporal Heterogeneity} Federated Learning (STHFL). Specially, the Global-Local Dynamic Prototype (GLDP) framework is designed for STHFL. In GLDP, the model in each client contains personalized layers which can dynamically adapt to different data distributions. For long-tailed data distribution, global prototypes are served as complementary knowledge for the training on classes with few samples in clients without leaking privacy. As tasks increase in clients, the knowledge of local prototypes generated in previous tasks guides for training in the current task to solve catastrophic forgetting. Meanwhile, the global-local prototypes are updated through the moving average method after training local prototypes in clients. Finally, we evaluate the effectiveness of GLDP, which achieves remarkable results compared to state-of-the-art methods in STHFL scenarios.

联邦学习时空异构长尾分布持续学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。