arXiv:2409.13645cs.LGcs.CR2024-09被引 2

通过个性化模型共享与平滑感知优化,提升隐私保护下的联邦学习性能。

DP$^2$-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimization

  • 采用局部模型共享与平滑感知优化,缓解噪声与裁剪带来的性能损失。
  • 在异构数据下相比现有方法显著提升准确率,隐私保障不变。
  • 适合注重隐私与模型性能平衡的分布式学习场景。

联邦学习(FL)是一种分布式机器学习方法,允许多个客户端在不共享原始数据的情况下协同训练模型。为防止通过共享的模型更新推断敏感信息,差分隐私联邦学习(DPFL)被提出。DPFL通过裁剪和添加随机噪声来保护隐私,但现有方法常导致模型性能严重下降,尤其在数据异构情况下。为此,本文提出新型DPFL方法DP²-FedSAM:结合个性化部分模型共享与平滑感知最小化优化器,有效缓解噪声添加与裁剪的负面影响,显著提升模型效用且不牺牲隐私。理论分析表明该方法具备严格的隐私与收敛性保证。基于常见基准数据集的实验验证了其在异构数据设置下优于现有方法的隐私-效用权衡能力。

原文摘要 · Abstract (English)

Federated learning (FL) is a distributed machine learning approach that allows multiple clients to collaboratively train a model without sharing their raw data. To prevent sensitive information from being inferred through the model updates shared in FL, differentially private federated learning (DPFL) has been proposed. DPFL ensures formal and rigorous privacy protection in FL by clipping and adding random noise to the shared model updates. However, the existing DPFL methods often result in severe model utility degradation, especially in settings with data heterogeneity. To enhance model utility, we propose a novel DPFL method named DP$^2$-FedSAM: Differentially Private and Personalized Federated Learning with Sharpness-Aware Minimization. DP$^2$-FedSAM leverages personalized partial model-sharing and sharpness-aware minimization optimizer to mitigate the adverse impact of noise addition and clipping, thereby significantly improving model utility without sacrificing privacy. From a theoretical perspective, we provide a rigorous theoretical analysis of the privacy and convergence guarantees of our proposed method. To evaluate the effectiveness of DP$^2$-FedSAM, we conduct extensive evaluations based on common benchmark datasets. Our results verify that our method improves the privacy-utility trade-off compared to the existing DPFL methods, particularly in heterogeneous data settings.

联邦学习差分隐私模型优化异构数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。