arXiv:2409.02189cs.LG2024-09被引 4

通过梯度分布差异识别低质量数据,提升联邦学习在噪声数据下的模型性能。

Collaboratively Learning Federated Models from Noisy Decentralized Data

  • 利用梯度范数分布差异判断客户端输入数据质量。
  • 在非独立同分布设置下提升模型性能达15.85%。
  • 可无缝接入现有联邦学习框架,适合数据噪声多的边缘场景。

联邦学习(FL)作为一种在边缘设备本地数据上协作训练模型的方法,已引起广泛关注,同时保持数据去中心化。然而,如何评估本地客户端贡献数据的质量仍是关键挑战,因为本地数据常受各类噪声和扰动影响,损害聚合过程并导致次优全局模型。本文聚焦于输入空间中的噪声问题,该领域相较于标签噪声研究较少。我们提出一种基于梯度空间中客户端输入的综合评估方法,其灵感来自在噪声与干净输入数据上训练的模型梯度范数分布密度的显著差异。基于此,我们设计了一种简单有效的初始阶段方法,用于识别数据质量低的客户端。此外,我们提出一种噪声感知的联邦聚合方法——联邦噪声筛选(FedNS),可作为插件与广泛使用的联邦学习策略结合使用。在多种基准数据集及不同联邦设置下的大量实验表明,该方法有效。当在噪声去中心化数据上学习时,FedNS可轻松集成至现有策略中,在独立同分布(IID)和非独立同分布(non-IID)设置下分别提升全局模型性能达13.68%和15.85%。

原文摘要 · Abstract (English)

Federated learning (FL) has emerged as a prominent method for collaboratively training machine learning models using local data from edge devices, all while keeping data decentralized. However, accounting for the quality of data contributed by local clients remains a critical challenge in FL, as local data are often susceptible to corruption by various forms of noise and perturbations, which compromise the aggregation process and lead to a subpar global model. In this work, we focus on addressing the problem of noisy data in the input space, an under-explored area compared to the label noise. We propose a comprehensive assessment of client input in the gradient space, inspired by the distinct disparity observed between the density of gradient norm distributions of models trained on noisy and clean input data. Based on this observation, we introduce a straightforward yet effective approach to identify clients with low-quality data at the initial stage of FL. Furthermore, we propose a noise-aware FL aggregation method, namely Federated Noise-Sifting (FedNS), which can be used as a plug-in approach in conjunction with widely used FL strategies. Our extensive evaluation on diverse benchmark datasets under different federated settings demonstrates the efficacy of FedNS. Our method effortlessly integrates with existing FL strategies, enhancing the global model's performance by up to 13.68% in IID and 15.85% in non-IID settings when learning from noisy decentralized data.

联邦学习噪声鲁棒梯度分析去中心化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。