分离特征提取与分类,提升联邦学习准确率与隐私保护。
FedFeat+: A Robust Federated Learning Framework Through Federated Aggregation and Differentially Private Feature-Based Classifier Retraining
- 分两阶段训练:本地提取特征,服务器聚合并重训分类器。
- 在多个数据集上提升准确率3.92%至12.34%,优于FedAvg。
- 结合差分隐私保护特征传输,适合隐私敏感场景使用。
本文提出FedFeat+框架,将特征提取与分类任务解耦。采用两阶段训练:本地训练后,客户端向服务器上传模型权重及最终轮次的特征;服务器使用FedAvg聚合模型,并利用共享特征重训全局分类器。该过程增强模型对数据分布的整体理解,提升跨数据集泛化能力,使分类器能自适应影响后续本地训练中的特征提取器。通过在共享特征向量中引入噪声,实现差分隐私保护,平衡准确率与隐私安全。理论分析证明了收敛性、性能提升与隐私保障。在CIFAR-10、CIFAR-100、MNIST和FMNIST等基准数据集上的实验证明,即使仅使用轻量级两层CNN分类器,FedFeat+在IID与非IID场景下均显著优于FedAvg,准确率提升达3.92%至12.34%。
原文摘要 · Abstract (English)
In this paper, we propose the FedFeat+ framework, which distinctively separates feature extraction from classification. We develop a two-tiered model training process: following local training, clients transmit their weights and some features extracted from the feature extractor from the final local epochs to the server. The server aggregates these models using the FedAvg method and subsequently retrains the global classifier utilizing the shared features. The classifier retraining process enhances the model's understanding of the holistic view of the data distribution, ensuring better generalization across diverse datasets. This improved generalization enables the classifier to adaptively influence the feature extractor during subsequent local training epochs. We establish a balance between enhancing model accuracy and safeguarding individual privacy through the implementation of differential privacy mechanisms. By incorporating noise into the feature vectors shared with the server, we ensure that sensitive data remains confidential. We present a comprehensive convergence analysis, along with theoretical reasoning regarding performance enhancement and privacy preservation. We validate our approach through empirical evaluations conducted on benchmark datasets, including CIFAR-10, CIFAR-100, MNIST, and FMNIST, achieving high accuracy while adhering to stringent privacy guarantees. The experimental results demonstrate that the FedFeat+ framework, despite using only a lightweight two-layer CNN classifier, outperforms the FedAvg method in both IID and non-IID scenarios, achieving accuracy improvements ranging from 3.92 % to 12.34 % across CIFAR-10, CIFAR-100, and Fashion-MNIST datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。