根据用户隐私偏好动态分配保护力度,提升联邦学习隐私与性能平衡
FedRE: Robust and Effective Federated Learning with Privacy Preference
- 按客户端对敏感信息的偏好,分层分配差分隐私预算
- 在两个数据集上实现与顶尖方法相当的检测准确率
- 适合注重隐私差异性且追求模型精度的应用场景
尽管联邦学习通过服务器端梯度聚合避免原始数据泄露,但客户端上传的梯度仍可能暴露隐私。现有方法普遍采用统一机制对每样本添加噪声以满足局部差分隐私(LDP),但未考虑不同客户端对敏感信息(PSI)的隐私偏好差异,导致对非敏感信息过度保护,引入冗余噪声并降低模型性能。本文提出FedRE,首先定义基于客户端隐私偏好的敏感信息(PSI),然后在层面上为高PSI梯度分配更小的隐私预算,实现对敏感信息更严格的隐私保护;同时设计基于扰动信息分布的参数聚合机制,缓解LDP带来的性能下降。在T-SROIE和DocTamper文本篡改检测任务上的实验表明,FedRE在保持强隐私保障的同时,达到与当前最优方法相当的性能。
原文摘要 · Abstract (English)
Despite Federated Learning (FL) employing gradient aggregation at the server for distributed training to prevent the privacy leakage of raw data, private information can still be divulged through the analysis of uploaded gradients from clients. Substantial efforts have been made to integrate local differential privacy (LDP) into the system to achieve a strict privacy guarantee. However, existing methods fail to take practical issues into account by merely perturbing each sample with the same mechanism while each client may have their own privacy preferences on privacy-sensitive information (PSI), which is not uniformly distributed across the raw data. In such a case, excessive privacy protection from private-insensitive information can additionally introduce unnecessary noise, which may degrade the model performance. In this work, we study the PSI within data and develop FedRE, that can simultaneously achieve robustness and effectiveness benefits with LDP protection. More specifically, we first define PSI with regard to the privacy preferences of each client. Then, we optimize the LDP by allocating less privacy budget to gradients with higher PSI in a layer-wise manner, thus providing a stricter privacy guarantee for PSI. Furthermore, to mitigate the performance degradation caused by LDP, we design a parameter aggregation mechanism based on the distribution of the perturbed information. We conducted experiments with text tamper detection on T-SROIE and DocTamper datasets, and FedRE achieves competitive performance compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。