FedCova通过特征协方差提升模型抗噪声能力,无需依赖外部数据。
FedCova: Robust Federated Covariance Learning Against Noisy Labels
- 基于特征协方差设计鲁棒编码目标,增强模型内在抗噪性。
- 在CIFAR-10/100和Clothing1M上显著优于现有方法。
- 适用于异构数据分布下的真实噪声场景,适合工业级联邦学习应用。
分布式数据中的噪声标签导致严重的局部过拟合并损害全局模型性能。现有方法多依赖于选择干净设备或对齐公开清洁数据集,而非提升模型自身鲁棒性。本文提出FedCova,一种无依赖的联邦协方差学习框架,通过新视角的特征协方差增强模型内在鲁棒性,消除对外部资源的依赖。具体而言,FedCova将数据编码至判别性强且抗噪的特征空间,基于互信息最大化,设计仅依赖类别特征协方差与误差容忍项的联邦有损特征编码目标。利用协方差表征的特征子空间,构建子空间增强型联邦分类器。FedCova统一了三个关键过程:网络训练、分类器构建与噪声标签修正。我们在异构数据分布下,于对称与非对称噪声设置中实现FedCova,实验结果在CIFAR-10/100与真实噪声数据集Clothing1M上均显著优于当前最优方法。
原文摘要 · Abstract (English)
Noisy labels in distributed datasets induce severe local overfitting and consequently compromise the global model in federated learning (FL). Most existing solutions rely on selecting clean devices or aligning with public clean datasets, rather than endowing the model itself with robustness. In this paper, we propose FedCova, a dependency-free federated covariance learning framework that eliminates such external reliances by enhancing the model's intrinsic robustness via a new perspective on feature covariances. Specifically, FedCova encodes data into a discriminative but resilient feature space to tolerate label noise. Built on mutual information maximization, we design a novel objective for federated lossy feature encoding that relies solely on class feature covariances with an error tolerance term. Leveraging feature subspaces characterized by covariances, we construct a subspace-augmented federated classifier. FedCova unifies three key processes through the covariance: (1) training the network for feature encoding, (2) constructing a classifier directly from the learned features, and (3) correcting noisy labels based on feature subspaces. We implement FedCova across both symmetric and asymmetric noisy settings under heterogeneous data distribution. Experimental results on CIFAR-10/100 and real-world noisy dataset Clothing1M demonstrate the superior robustness of FedCova compared with the state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。