arXiv:2506.00932cs.LG2025-06

解决低数据联邦学习中模型更新停滞问题,提升协作效率。

Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity

  • 通过周期性引入瞬时稀疏性,激活深层模型更新
  • 在多个数据集和架构上显著降低层间惯性现象
  • 适合资源受限环境下的高效联邦学习应用

联邦学习(FL)可在保护数据隐私的前提下实现跨分散客户端的协同模型训练,依赖聚合更新构建稳健的全局模型。然而,由于数据异质性和本地数据量有限,有效协作常受阻碍。本文发现,在多种联邦学习场景中,全局模型中间层在早期通信轮次后几乎不再更新,形成层间惯性现象,严重制约全局聚合效果。为此,我们提出LIPS(Layer-wise Inertia Phenomenon with Sparsity),一种简单有效的策略,通过周期性引入瞬时稀疏性,激发有意义的模型更新,增强聚合能力。实验表明,LIPS能有效缓解层间惯性,提升各类联邦学习场景下的整体性能。本工作不仅深化了对联邦学习中层级学习动态的理解,也为资源受限环境下的高效协作提供了新路径。代码已公开于:https://github.com/QiaoXiao7282/LIPS。

原文摘要 · Abstract (English)

Federated learning (FL) enables collaborative model training across decentralized clients while preserving data privacy, leveraging aggregated updates to build robust global models. However, this training paradigm faces significant challenges due to data heterogeneity and limited local datasets, which often impede effective collaboration. In such scenarios, we identify the Layer-wise Inertia Phenomenon in FL, wherein the middle layers of global model undergo minimal updates after early communication rounds, ultimately limiting the effectiveness of global aggregation. We demonstrate the presence of this phenomenon across a wide range of federated settings, spanning diverse datasets and architectures. To address this issue, we propose LIPS (Layer-wise Inertia Phenomenon with Sparsity), a simple yet effective method that periodically introduces transient sparsity to stimulate meaningful updates and empower global aggregation. Experiments demonstrate that LIPS effectively mitigates layer-wise inertia, enhances aggregation effectiveness, and improves overall performance in various FL scenarios. This work not only deepens the understanding of layer-wise learning dynamics in FL but also paves the way for more effective collaboration strategies in resource-constrained environments. Our code is publicly available at: https://github.com/QiaoXiao7282/LIPS.

联邦学习模型优化稀疏性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。