用超网络生成本地模型参数,避免直接上传梯度,防梯度反演攻击。
A New Federated Learning Framework Against Gradient Inversion Attacks
- 用超网络生成模型参数,仅上传超网络参数
- 实验表明隐私保护效果强且性能接近传统联邦学习
- 适合对数据隐私要求高的场景如医疗、金融
联邦学习(FL)通过客户端在不共享原始数据的情况下协同训练模型来保护数据隐私。然而,近期研究发现,FL过程中交换的信息易受梯度反演攻击(GIA),因此已有多种隐私保护方法被引入FL,如安全多方计算(SMC)、同态加密(HE)和差分隐私(DP)。尽管这些方法能保护隐私,但不可避免地带来隐私-效用权衡。本文重新审视了在GIA下隐私泄露的关键——频繁共享包含私有数据的模型梯度,提出一种新思路:通过设计新型隐私保护联邦学习框架,切断共享参数与本地私有数据之间的直接关联。具体而言,提出超网络联邦学习(HyperFL)框架,利用超网络生成本地模型参数,仅将超网络参数上传至服务器进行聚合。理论分析证明了所提方法的收敛速度,大量实验结果验证了其在隐私保护方面的有效性及与传统方法相当的性能表现。代码已开源于 https://github.com/Pengxin-Guo/HyperFL。
原文摘要 · Abstract (English)
Federated Learning (FL) aims to protect data privacy by enabling clients to collectively train machine learning models without sharing their raw data. However, recent studies demonstrate that information exchanged during FL is subject to Gradient Inversion Attacks (GIA) and, consequently, a variety of privacy-preserving methods have been integrated into FL to thwart such attacks, such as Secure Multi-party Computing (SMC), Homomorphic Encryption (HE), and Differential Privacy (DP). Despite their ability to protect data privacy, these approaches inherently involve substantial privacy-utility trade-offs. By revisiting the key to privacy exposure in FL under GIA, which lies in the frequent sharing of model gradients that contain private data, we take a new perspective by designing a novel privacy preserve FL framework that effectively ``breaks the direct connection'' between the shared parameters and the local private data to defend against GIA. Specifically, we propose a Hypernetwork Federated Learning (HyperFL) framework that utilizes hypernetworks to generate the parameters of the local model and only the hypernetwork parameters are uploaded to the server for aggregation. Theoretical analyses demonstrate the convergence rate of the proposed HyperFL, while extensive experimental results show the privacy-preserving capability and comparable performance of HyperFL. Code is available at https://github.com/Pengxin-Guo/HyperFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。