针对传感器环境中的遗忘问题,提出动态调整训练频率的联邦学习方法。
FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments
- 根据数据分布变化和设备能力动态调节离线训练频率,减少遗忘。
- 在真实人体活动识别场景中,提升10%~15%训练效率,收敛更快更稳。
- 适用于资源受限、用户参与不稳定的物联网健康监测等场景。
联邦学习(FL)通过共享模型更新而非原始数据,在保护隐私的同时实现协同建模。本文聚焦于普遍存在计算环境(如人体活动识别,HAR),其特点包括资源受限的终端设备、持续流式传感器数据以及间歇性客户端参与。用户行为差异导致数据分布非平稳,现有联邦学习方法因假设不符而面临挑战。数据异质性与间歇参与共同引发严重灾难性遗忘(CF)问题。不同于连续学习中使用记忆回放的方法,联邦学习受隐私限制无法采用此类策略。为此,本文提出FlexFed,一种新型联邦学习方法:优先保留关键数据以高效利用内存,并根据分布漂移、客户端能力及离线时长动态调整离线训练频率。为更准确评估遗忘,引入新度量指标,考虑欠代表数据的影响。构建基于真实HAR的评估框架,模拟流式数据、动态分布、数据不平衡及不同可用性。实验表明,FlexFed显著缓解遗忘,提升10%~15%训练效率,对不常出现或欠代表数据实现更快、更稳定的收敛。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training while preserving privacy by allowing clients to share model updates instead of raw data. Pervasive computing environments (e.g., for Human Activity Recognition, HAR), which we focus on in this paper, are characterized by resource-constrained end devices, streaming sensor data and intermittent client participation. Variations in user behavior, common in HAR environments, often result in non-stationary data distributions. As such, existing FL approaches face challenges in HAR settings due to differing assumptions. The combined effects of HAR characteristics, namely heterogeneous data and intermittent participation, can lead to a severe issue called catastrophic forgetting (CF). Unlike Continuous Learning (CL), which addresses CF using memory and replay mechanisms, FL's privacy constraints prohibit such strategies. To tackle CF in HAR environments, we propose FlexFed, a novel FL approach that prioritizes data retention for efficient memory use and dynamically adjusts offline training frequency based on distribution shifts, client capability and offline duration. To better quantify CF in FL, we introduce a new metric that accounts for under-represented data, enabling more accurate evaluations. We also develop a realistic HAR-based evaluation framework that simulates streaming data, dynamic distributions, imbalances and varying availability. Experiments show that FlexFed mitigates CF more effectively, improves FL efficiency by 10 to 15 % and achieves faster, more stable convergence, especially for infrequent or under-represented data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。