用CLIP在无监督联邦学习中实现分类,解决标签分布不均问题。
Cooperative Pseudo Labeling for Unsupervised Federated Classification
- 客户端上传伪标签分布,服务器动态调整以缓解类别不平衡。
- 视觉提示全局聚合,文本提示本地保留,兼顾协作与个性化。
- 首次将CLIP用于联邦分类,适合跨设备图像识别场景。
无监督联邦学习(UFL)旨在不共享数据或标签的情况下,协同训练全局模型。以往研究主要集中在表示学习和聚类任务。近期,视觉语言模型(如CLIP)因其强大的零样本预测能力受到关注,使原本在UFL范式下难以实现的分类任务迎来新机遇,但尚未被充分探索。本文首次将CLIP引入联邦分类,提出联邦协同伪标签方法(FedCoPL)。客户端估计并上传伪标签分布,服务器进行校正并重新分发,避免全局类别失衡。此外,设计部分提示聚合协议:视觉提示(包含通用图像特征)在服务器端聚合,而编码个性化知识的文本提示保留在本地。大量实验表明,该方法显著优于基线。代码已开源。
原文摘要 · Abstract (English)
Unsupervised Federated Learning (UFL) aims to collaboratively train a global model across distributed clients without sharing data or accessing label information. Previous UFL works have predominantly focused on representation learning and clustering tasks. Recently, vision language models (e.g., CLIP) have gained significant attention for their powerful zero-shot prediction capabilities. Leveraging this advancement, classification problems that were previously infeasible under the UFL paradigm now present promising new opportunities, yet remain largely unexplored. In this paper, we extend UFL to the classification problem with CLIP for the first time and propose a novel method, \underline{\textbf{Fed}}erated \underline{\textbf{Co}}operative \underline{\textbf{P}}seudo \underline{\textbf{L}}abeling (\textbf{FedCoPL}). Specifically, clients estimate and upload their pseudo label distribution, and the server adjusts and redistributes them to avoid global imbalance among classes. Moreover, we introduce a partial prompt aggregation protocol for effective collaboration and personalization. In particular, visual prompts containing general image features are aggregated at the server, while text prompts encoding personalized knowledge are retained locally. Extensive experiments demonstrate the superior performance of our FedCoPL compared to baseline methods. Our code is available at \href{https://github.com/krumpguo/FedCoPL}{https://github.com/krumpguo/FedCoPL}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。