解决联邦半监督学习中标签极少导致的性能差距问题
(FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning
- 用锐度感知一致性正则化筛选关键无标签样本
- 在三个数据集上显著缩小与集中式半监督学习的性能差距
- 适合标签稀缺且需保护隐私的分布式场景
联邦学习（FL）是一种分布式机器学习框架，可在保护客户端隐私数据的前提下训练高精度全局模型。然而，多数FL方法假设客户端拥有标注数据，这在实际中并不常见。联邦半监督学习（FSSL）旨在解决标签不足问题，适用于仅服务器拥有少量标注数据而客户端无标注数据的情形。但目前FSSL与集中式半监督学习（SSL）之间存在显著性能差距，其根源在于确认偏误，该偏误在FSSL中因多轮本地训练和标注/未标注数据分离而更加突出。本文提出$(FL)^2$，一种针对无标签客户端的鲁棒训练方法，采用锐度感知一致性正则化。我们发现直接对原始伪标签损失进行正则化效果不佳，因此精心选择无标签样本用于正则化。进一步引入客户端自适应阈值和学习状态感知聚合机制，根据各客户端的学习进度动态调整训练过程。在三个基准数据集上的实验表明，该方法显著提升性能，有效缩小与SSL的差距，尤其在标签极为稀缺的情况下表现突出。
原文摘要 · Abstract (English)
Federated Learning (FL) is a distributed machine learning framework that trains accurate global models while preserving clients' privacy-sensitive data. However, most FL approaches assume that clients possess labeled data, which is often not the case in practice. Federated Semi-Supervised Learning (FSSL) addresses this label deficiency problem, targeting situations where only the server has a small amount of labeled data while clients do not. However, a significant performance gap exists between Centralized Semi-Supervised Learning (SSL) and FSSL. This gap arises from confirmation bias, which is more pronounced in FSSL due to multiple local training epochs and the separation of labeled and unlabeled data. We propose $(FL)^2$, a robust training method for unlabeled clients using sharpness-aware consistency regularization. We show that regularizing the original pseudo-labeling loss is suboptimal, and hence we carefully select unlabeled samples for regularization. We further introduce client-specific adaptive thresholding and learning status-aware aggregation to adjust the training process based on the learning progress of each client. Our experiments on three benchmark datasets demonstrate that our approach significantly improves performance and bridges the gap with SSL, particularly in scenarios with scarce labeled data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。