通过剔除标注噪声严重的客户端提升联邦学习性能
Federated Learning Client Pruning for Noisy Labels
- 根据客户端在干净验证集上的表现评分,筛选并剔除噪声客户端
- 在多种噪声水平下均实现更高准确率、更快收敛和更低通信开销
- 适合边缘设备上存在标签噪声的联邦学习场景
联邦学习(FL)可在保护数据隐私的前提下实现分布式设备间的协同建模,但现有方法多假设数据标注清洁,这在资源受限的边缘设备中并不现实。实际上,标签噪声普遍存在,严重制约了联邦学习的表现。已有方法尝试进行标签修正和鲁棒训练,但在高噪声环境下效果有限。本文提出ClipFL(客户端剪枝的联邦学习),从新视角应对标签噪声问题。ClipFL通过跟踪一个噪声候选分数(NCS)来识别并剔除表现异常的客户端。该框架包含三个阶段:预剪枝阶段识别潜在噪声客户端并计算其NCS;客户端剪枝阶段剔除NCS最高的部分客户端;后剪枝阶段在剩余清洁客户端上使用标准联邦学习进行模型微调。实证评估表明,ClipFL在多种数据集和噪声水平下均表现优异,能准确识别噪声客户端,相比现有最优方法显著提升性能、加速收敛并降低通信成本。代码已公开于https://github.com/MMorafah/ClipFL。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across decentralized edge devices while preserving data privacy. However, existing FL methods often assume clean annotated datasets, impractical for resource-constrained edge devices. In reality, noisy labels are prevalent, posing significant challenges to FL performance. Prior approaches attempt label correction and robust training techniques but exhibit limited efficacy, particularly under high noise levels. This paper introduces ClipFL (Federated Learning Client Pruning), a novel framework addressing noisy labels from a fresh perspective. ClipFL identifies and excludes noisy clients based on their performance on a clean validation dataset, tracked using a Noise Candidacy Score (NCS). The framework comprises three phases: pre-client pruning to identify potential noisy clients and calculate their NCS, client pruning to exclude a percentage of clients with the highest NCS, and post-client pruning for fine-tuning the global model with standard FL on clean clients. Empirical evaluation demonstrates ClipFL's efficacy across diverse datasets and noise levels, achieving accurate noisy client identification, superior performance, faster convergence, and reduced communication costs compared to state-of-the-art FL methods. Our code is available at https://github.com/MMorafah/ClipFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。