针对联邦学习中的噪声标签问题,提出动态停止与损失修正双重策略。
FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels
- 通过动态早停防止模型过拟合错误标签
- 设计适配联邦学习的损失修正机制,提升在异构数据下的性能
- 理论证明噪声标签下目标函数可逼近干净标签分布,适合实际部署
联邦学习(FL)是一种保护隐私的分布式学习框架,允许多个客户端在不共享原始数据的情况下协同训练全局模型。然而,由于数据分布异质性和通信限制,处理噪声标签仍是重大挑战,会严重降低模型性能。为此,我们提出FedEFC,一种新型方法以缓解噪声标签的影响。该方法通过两个关键技术实现:(1) 预早停,通过动态确定最优训练终止点,防止对误标数据过拟合;(2) 损失修正,调整模型更新以应对标签噪声。特别地,我们设计了一种适用于联邦学习独特挑战(如数据异质性与去中心化训练)的有效损失修正方案。此外,我们基于复合适当损失性质提供理论分析,证明在噪声标签分布下,联邦学习的目标函数可与干净标签分布对齐。大量实验结果验证了该方法的有效性,在异构数据设置下显著优于现有技术,例如相比现有损失修正方法相对性能提升最高达41.64%。
原文摘要 · Abstract (English)
Federated Learning (FL) is a powerful framework for privacy-preserving distributed learning. It enables multiple clients to collaboratively train a global model without sharing raw data. However, handling noisy labels in FL remains a major challenge due to heterogeneous data distributions and communication constraints, which can severely degrade model performance. To address this issue, we propose FedEFC, a novel method designed to tackle the impact of noisy labels in FL. FedEFC mitigates this issue through two key techniques: (1) prestopping, which prevents overfitting to mislabeled data by dynamically halting training at an optimal point, and (2) loss correction, which adjusts model updates to account for label noise. In particular, we develop an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training. Furthermore, we provide a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution. Extensive experimental results validate the effectiveness of our approach, showing that it consistently outperforms existing FL techniques in mitigating the impact of noisy labels, particularly under heterogeneous data settings (e.g., achieving up to 41.64% relative performance improvement over the existing loss correction method).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。