针对不同隐私需求的客户端,提出更高效的联邦学习隐私保护方法。
Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacy
- 按隐私预算分组客户端,组内实现细粒度差分隐私
- 结合模型稀疏化与采样优化,降低噪声并提升模型效果
- 理论支撑强,适合高隐私需求且异构的现实场景
具有客户端级差分隐私(DP)的联邦学习为协同训练模型提供了强有力的隐私保护框架。然而,传统方法如DP-FedAvg在客户端隐私要求异构时表现不佳,因需对所有客户端统一采用最严格的隐私水平,导致过度添加DP噪声,显著降低模型性能。现有改进方法多假设可信服务器且依赖启发式策略,缺乏理论基础,表现不优。本文在客户端与服务器均为诚实但好奇的实用攻击模型下,提出GDPFed:根据客户端隐私预算分组,组内实现客户端级差分隐私,减少隐私预算浪费,提升模型效用。基于对GDPFed的隐私与收敛性分析,发现DP噪声大小取决于模型维度及各组采样比例。为进一步优化,引入GDPFed$^+$,融合模型稀疏化以消除冗余噪声,并优化各组采样比例以最小化收敛误差。在多个基准数据集上的大量实验表明,GDPFed$^+$相比现有最优方法有显著性能提升。
原文摘要 · Abstract (English)
Federated Learning with client-level differential privacy (DP) provides a promising framework for collaboratively training models while rigorously protecting clients' privacy. However, classic approaches like DP-FedAvg struggle when clients have heterogeneous privacy requirements, as they must uniformly enforce the strictest privacy level across all clients, leading to excessive DP noise and significant degradation in model utility. Existing methods to improve the model utility in such heterogeneous privacy settings often assume a trusted server and are largely heuristic, resulting in suboptimal performance and lacking strong theoretical foundations. In this work, we address these challenges under a practical attack model where both clients and the server are honest-but-curious. We propose GDPFed, which partitions clients into groups based on their privacy budgets and achieves client-level DP within each group to reduce the privacy budget waste and hence improve the model utility. Based on the privacy and convergence analysis of GDPFed, we find that the magnitude of DP noise depends on both model dimensionality and the per-group client sampling ratios. To further improve the performance of GDPFed, we introduce GDPFed$^+$, which integrates model sparsification to eliminate unnecessary noise and optimizes per-group client sampling ratios to minimize convergence error. Extensive empirical evaluations on multiple benchmark datasets demonstrate the effectiveness of GDPFed$^+$, showing substantial performance gains compared with state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。