提出局部相似性约束,提升噪声标签下的联邦学习性能。
Local K-Similarity Constraint for Federated Learning with Label Noise
- 通过约束本地数据点间相似性实现客户端模型正则化
- 在多个视觉与医疗图像分类任务中超越现有最优方法
- 无需共享预训练模型架构,适用性强且通信成本低
在存在标签噪声的客户端上进行联邦学习是一个挑战,因为这些客户端可能污染全局模型,影响系统整体泛化能力。现有方法通常假设存在足够数量的干净标签客户端,可用于训练鲁棒全局模型并抑制噪声客户端的影响,但在大量异构客户端均含噪声标签时,该假设失效。此时需在客户端与全局模型通信前进行本地正则化,以防止全局模型被污染。虽然预训练自监督模型可有效用于本地正则化,但依赖预训练初始化的集中式方法在联邦设置中不切实际,因其模型过大,增加通信开销。为此,我们提出一种正则化目标,通过解耦预训练模型与分类模型,强制客户端内部相近数据点保持相似性。利用自监督预训练模型的表示空间评估样本间接近程度。该正则化方法在标准下游任务目标函数下应用,显著提升性能,在多个计算机视觉和医学图像分类基准上优于现有最先进联邦学习方法。不同于依赖自监督预训练初始化的技术,本方法不要求预训练模型与分类骨干网络共享架构,具备架构无关性。
原文摘要 · Abstract (English)
Federated learning on clients with noisy labels is a challenging problem, as such clients can infiltrate the global model, impacting the overall generalizability of the system. Existing methods proposed to handle noisy clients assume that a sufficient number of clients with clean labels are available, which can be leveraged to learn a robust global model while dampening the impact of noisy clients. This assumption fails when a high number of heterogeneous clients contain noisy labels, making the existing approaches ineffective. In such scenarios, it is important to locally regularize the clients before communication with the global model, to ensure the global model isn't corrupted by noisy clients. While pre-trained self-supervised models can be effective for local regularization, existing centralized approaches relying on pretrained initialization are impractical in a federated setting due to the potentially large size of these models, which increases communication costs. In that line, we propose a regularization objective for client models that decouples the pre-trained and classification models by enforcing similarity between close data points within the client. We leverage the representation space of a self-supervised pretrained model to evaluate the closeness among examples. This regularization, when applied with the standard objective function for the downstream task in standard noisy federated settings, significantly improves performance, outperforming existing state-of-the-art federated methods in multiple computer vision and medical image classification benchmarks. Unlike other techniques that rely on self-supervised pretrained initialization, our method does not require the pretrained model and classifier backbone to share the same architecture, making it architecture-agnostic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。