通过特征几何结构识别噪声标签,提升异构联邦学习性能
FedRG: Unleashing the Representation Geometry for Federated Learning with Noisy Clients
- 用自监督构建无标签球形表示,基于几何分布识别噪声
- 在多种噪声客户端下准确率最高达89.2%,显著优于现有方法
- 适合存在标注噪声的分布式训练场景,尤其适用于异构数据
联邦学习因分布式场景中不可避免的噪声标注导致性能下降。现有方法依赖损失值区分噪声样本,但在异构环境下可靠性不足。本文提出FedRG(联邦学习中的表示几何优先),从表示视角重构噪声识别范式:首先通过自监督生成无标签球形表示;然后迭代拟合球面von Mises-Fisher混合模型,捕捉语义聚类;结合语义-标签软映射机制,计算无标签与标注特征空间间的分布差异,从而稳健识别噪声样本,并更新模型;最后引入个性化噪声吸收矩阵实现鲁棒优化。大量实验表明,该方法在不同噪声客户端场景下均显著超越当前最优水平,最高准确率达89.2%。
原文摘要 · Abstract (English)
Federated learning (FL) suffers from performance degradation due to the inevitable presence of noisy annotations in distributed scenarios. Existing approaches have advanced in distinguishing noisy samples from the dataset for label correction by leveraging loss values. However, noisy samples recognition relying on scalar loss lacks reliability for FL under heterogeneous scenarios. In this paper, we rethink this paradigm from a representation perspective and propose \method~(\textbf{Fed}erated under \textbf{R}epresentation \textbf{G}emometry), which follows \textbf{the principle of ``representation geometry priority''} to recognize noisy labels. Firstly, \method~creates label-agnostic spherical representations by using self-supervision. It then iteratively fits a spherical von Mises-Fisher (vMF) mixture model to this geometry using previously identified clean samples to capture semantic clusters. This geometric evidence is integrated with a semantic-label soft mapping mechanism to derive a distribution divergence between the label-free and annotated label-conditioned feature space, which robustly identifies noisy samples and updates the vMF mixture model with the newly separated clean dataset. Lastly, we employ an additional personalized noise absorption matrix on noisy labels to achieve robust optimization. Extensive experimental results demonstrate that \method~significantly outperforms state-of-the-art methods for FL with data heterogeneity under diverse noisy clients scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。