基于置信度聚类的个性化联邦学习,提升隐私保护下的模型精度与通信效率。
TPFL: Tsetlin-Personalized Federated Learning with Confidence-Based Clustering
- 按客户端对特定类别的置信度聚类,只共享高置信权重。
- 在非独立同分布数据下,准确率最高达98.94%(MNIST)。
- 适合隐私敏感场景,尤其适用于资源受限的边缘设备。
机器学习领域快速发展,深度学习占据主流。然而,随着新算法如Tsetlin Machine(TM)的出现,探索替代方法的兴趣日益增长。联邦学习(FL)作为保护用户隐私的重要范式,近年来引入个性化技术以提升模型精度。本文提出新型方法TPFL:基于置信度聚类的个性化联邦学习,将模型按对特定类别的置信度分组。该方法使客户端仅共享高置信度的权重,避免因数据不足导致的错误参数聚合,特别适用于非独立同分布(non-IID)数据。同时,仅传递特定类别权重显著降低通信开销。与6种基线方法(FedAvg、FedProx、FLIS DC、FLIS HC、IFCA、FedTM)对比,TPFL在MNIST上达到98.94%准确率,FashionMNIST上为98.52%,FEMNIST上为91.16%。
原文摘要 · Abstract (English)
The world of Machine Learning (ML) has witnessed rapid changes in terms of new models and ways to process users data. The majority of work that has been done is focused on Deep Learning (DL) based approaches. However, with the emergence of new algorithms such as the Tsetlin Machine (TM) algorithm, there is growing interest in exploring alternative approaches that may offer unique advantages in certain domains or applications. One of these domains is Federated Learning (FL), in which users privacy is of utmost importance. Due to its novelty, FL has seen a surge in the incorporation of personalization techniques to enhance model accuracy while maintaining user privacy under personalized conditions. In this work, we propose a novel approach called TPFL: Tsetlin-Personalized Federated Learning, in which models are grouped into clusters based on their confidence towards a specific class. In this way, clustering can benefit from two key advantages. Firstly, clients share only what they are confident about, resulting in the elimination of wrongful weight aggregation among clients whose data for a specific class may have not been enough during the training. This phenomenon is prevalent when the data are non-Independent and Identically Distributed (non-IID). Secondly, by sharing only weights towards a specific class, communication cost is substantially reduced, making TPLF efficient in terms of both accuracy and communication cost. The TPFL results were compared with 6 other baseline methods; namely FedAvg, FedProx, FLIS DC, FLIS HC, IFCA and FedTM. The results demonstrated that TPFL performance better than baseline methods with 98.94% accuracy on MNIST, 98.52% accuracy on FashionMNIST and 91.16% accuracy on FEMNIST dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。