通过检测模型置信度差异，动态修正伪标签提升联邦半监督学习效果
Mind the Gap: Confidence Discrepancy Can Guide Federated Semi-Supervised Learning Across Pseudo-Mismatch
- 基于本地与全局模型置信度差异动态修正伪标签
- 在非独立同分布数据下提升准确率最高达8.2个百分点
- 适合标签稀缺且数据异构的联邦学习场景
联邦半监督学习（FSSL）旨在利用客户端有限标注数据和大量未标注数据训练具备强泛化能力的全局模型。现有FSSL方法依赖一致性正则化生成硬伪标签作为监督信号，但本文发现数据异构性会显著恶化伪标签质量。研究揭示：(1) 异构性加剧伪标签不匹配，导致模型性能下降与收敛变慢；(2) 随着异构性增强，本地与全局模型预测倾向出现明显分歧。为此，提出SAGE方法——基于置信度差异灵活修正伪标签，有效缓解错误伪标签带来的性能退化，并增强本地与全局模型的一致性。实验表明，SAGE在多个数据集上均优于现有FSSL方法，在性能与收敛速度上均有显著提升。代码已开源。
原文摘要 · Abstract (English)
Federated Semi-Supervised Learning (FSSL) aims to leverage unlabeled data across clients with limited labeled data to train a global model with strong generalization ability. Most FSSL methods rely on consistency regularization with pseudo-labels, converting predictions from local or global models into hard pseudo-labels as supervisory signals. However, we discover that the quality of pseudo-label is largely deteriorated by data heterogeneity, an intrinsic facet of federated learning. In this paper, we study the problem of FSSL in-depth and show that (1) heterogeneity exacerbates pseudo-label mismatches, further degrading model performance and convergence, and (2) local and global models' predictive tendencies diverge as heterogeneity increases. Motivated by these findings, we propose a simple and effective method called Semi-supervised Aggregation for Globally-Enhanced Ensemble (SAGE), that can flexibly correct pseudo-labels based on confidence discrepancies. This strategy effectively mitigates performance degradation caused by incorrect pseudo-labels and enhances consensus between local and global models. Experimental results demonstrate that SAGE outperforms existing FSSL methods in both performance and convergence. Our code is available at https://github.com/Jay-Codeman/SAGE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。