arXiv:2412.13589cs.LGcs.AI2024-12AAAI被引 3

提出首个面向半监督场景的去中心化联邦学习方法,提升无标签数据利用效率。

SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning

  • 通过邻域信息优化伪标签质量,增强无标签数据利用。
  • 设计基于共识的扩散模型生成合成数据,与伪标签数据混合训练。
  • 自适应聚合机制结合合成数据精度,适用于多类数据分布场景。

去中心化联邦学习(DFL)在无需中央服务器的情况下实现客户端间协作训练,缓解通信瓶颈并消除单点故障问题。现有大部分DFL研究聚焦于有监督学习,假设每个客户端具备充足标注数据。然而现实中大量数据为无标签。本文针对这一挑战,提出首个面向半监督学习(SSL)的去中心化联邦学习框架SemiDFL,支持客户端具有不同数据来源:部分仅有少量标注数据,部分完全无标签,部分则两者兼具。SemiDFL通过在数据与模型空间建立共识,显著提升性能。具体地,利用邻域信息改进伪标签质量,进而设计基于共识的扩散模型生成合成数据,与伪标签数据结合构建混合数据集。同时,提出自适应聚合策略,借助合成数据的模型精度进一步优化性能。大量实验表明,该方法在IID与非IID半监督场景下均显著优于现有中心化与去中心化联邦学习方案。

原文摘要 · Abstract (English)

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semisupervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabeled data. We then design a consensusbased diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both IID and non-IID SSL scenarios.

去中心化半监督联邦学习伪标签

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。