提出三种垂直联邦学习下数据移除方法,无需通信即可高效擦除用户、特征和样本。
Unlearning Clients, Features and Samples in Vertical Federated Learning
- 基于知识蒸馏与梯度上升,分别实现客户端、特征和样本的可验证移除
- 在6个表格与2个图像数据集上性能接近或优于重训练,损失仅1-5%
- 无需被动方参与通信,但需主动方保存历史嵌入向量
联邦学习(FL)已成为一种重要的分布式学习范式。为满足隐私法规如GDPR中用户要求删除其贡献的权利,服务端需具备从模型中移除特定信息的能力。本文研究垂直联邦学习(VFL)中的三种移除场景:移除客户端、移除特征、移除样本。针对前两者提出基于知识蒸馏的VFU-KD方法,针对后者提出基于梯度上升的VFU-GA方法。通过成员推理攻击(MIA)验证近似移除效果。实验在六个表格数据集和两个图像数据集上表明,两种方法在多数情况下性能与或优于从头重训练及基准方法R2S,提升0-2%;其余情况仅产生1-5%的性能损失。相比现有方法,本方案在移除过程中无需主动方与被动方通信,但要求主动方存储先前传输的嵌入向量。
原文摘要 · Abstract (English)
Federated Learning (FL) has emerged as a prominent distributed learning paradigm. Within the scope of privacy preservation, information privacy regulations such as GDPR entitle users to request the removal (or unlearning) of their contribution from a service that is hosting the model. For this purpose, a server hosting an ML model must be able to unlearn certain information in cases such as copyright infringement or security issues that can make the model vulnerable or impact the performance of a service based on that model. While most unlearning approaches in FL focus on Horizontal FL (HFL), where clients share the feature space and the global model, Vertical FL (VFL) has received less attention from the research community. VFL involves clients (passive parties) sharing the sample space among them while not having access to the labels. In this paper, we explore unlearning in VFL from three perspectives: unlearning clients, unlearning features, and unlearning samples. To unlearn clients and features we introduce VFU-KD which is based on knowledge distillation (KD) while to unlearn samples, VFU-GA is introduced which is based on gradient ascent. To provide evidence of approximate unlearning, we utilize Membership Inference Attack (MIA) to audit the effectiveness of our unlearning approach. Our experiments across six tabular datasets and two image datasets demonstrate that VFU-KD and VFU-GA achieve performance comparable to or better than both retraining from scratch and the benchmark R2S method in many cases, with improvements of $(0-2\%)$. In the remaining cases, utility scores remain comparable, with a modest utility loss ranging from $1-5\%$. Unlike existing methods, VFU-KD and VFU-GA require no communication between active and passive parties during unlearning. However, they do require the active party to store the previously communicated embeddings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。