首次实现差分隐私联邦学习中的数据遗忘,利用噪声提升隐私保护效果。
Upcycling Noise for Federated Unlearning
- 通过局部模型回滚与全局噪声校准实现遗忘
- 在四个真实数据集上性能优于主流遗忘方法
- 适合需要满足数据删除权的联邦学习场景
在联邦学习(FL)中,多个客户端协作训练模型而不共享原始数据。该范式可通过差分隐私(DP)进一步增强以抵御信息推断攻击,称为DPFL。新兴的隐私需求——客户端的“被遗忘权”——为DPFL带来新挑战,但尚未得到充分研究。尽管已有大量关于联邦遗忘(FU)的研究,但由于DP机制引入的噪声会削弱其有效性与效率,现有方法不适用于DPFL。本文首次提出具有不可区分性的联邦遗忘(FUI),实现对DPFL中目标客户端本地数据的遗忘。FUI包含两个主要步骤:局部模型回滚与全局噪声校准,使遗忘模型在统计上无法与重新训练模型区分。我们证明,DPFL中添加的噪声可在局部模型回滚后赋予遗忘模型一定级别的不可区分性,并通过全局噪声校准进一步强化遗忘程度。此外,为高效一致地实现所提FUI,我们构建两阶段斯塔克尔伯格博弈,推导出服务器与目标客户端的最优遗忘策略。隐私与收敛性分析提供了理论保障,基于四个真实数据集的实验结果表明,所提FUI在模型性能和效率方面均优于主流FU方案。仿真结果进一步验证了所获遗忘策略的最优性。
原文摘要 · Abstract (English)
In Federated Learning (FL), multiple clients collaboratively train a model without sharing raw data. This paradigm can be further enhanced by Differential Privacy (DP) to protect local data from information inference attacks and is thus termed DPFL. An emerging privacy requirement, ``the right to be forgotten'' for clients, poses new challenges to DPFL but remains largely unexplored. Despite numerous studies on federated unlearning (FU), they are inapplicable to DPFL because the noise introduced by the DP mechanism compromises their effectiveness and efficiency. In this paper, we propose Federated Unlearning with Indistinguishability (FUI) to unlearn the local data of a target client in DPFL for the first time. FUI consists of two main steps: local model retraction and global noise calibration, resulting in an unlearning model that is statistically indistinguishable from the retrained model. Specifically, we demonstrate that the noise added in DPFL can endow the unlearning model with a certain level of indistinguishability after local model retraction, and then fortify the degree of unlearning through global noise calibration. Additionally, for the efficient and consistent implementation of the proposed FUI, we formulate a two-stage Stackelberg game to derive optimal unlearning strategies for both the server and the target client. Privacy and convergence analyses confirm theoretical guarantees, while experimental results based on four real-world datasets illustrate that our proposed FUI achieves superior model performance and higher efficiency compared to mainstream FU schemes. Simulation results further verify the optimality of the derived unlearning strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。