提出新方法实现垂直联邦学习中数据与标签的高效删除。
Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning
- 用对偶优化框架解决跨方数据协同删除难题
- 删除后模型性能损失小,计算通信开销更低
- 适合需要隐私合规的医疗、金融等场景
联邦遗忘已成为应对协作机器学习中隐私问题的有力手段,尤其在敏感数据被模型记忆时。它可移除特定数据对训练模型的影响,契合日益重要的“被遗忘权”。尽管横向联邦学习中已有广泛研究,但因特征分布式架构,垂直联邦学习(VFL)中的遗忘仍具挑战性。VFL遗忘包括样本遗忘(移除特定数据点影响)和标签遗忘(移除整个类别)。由于不同参与方持有同一样本的互补特征,遗忘任务需跨方协调,带来计算开销与特征依赖复杂性。为此,我们提出FedORA(Federated Optimization for data Removal via primal-dual Algorithm),用于VFL中的样本与标签遗忘。将特定样本或标签的移除建模为带约束的优化问题,通过原始-对偶框架求解。引入新遗忘损失函数,促进分类不确定性而非错误分类;自适应步长提升稳定性;非对称批处理设计,区分遗忘数据与保留数据的先验影响,有效降低计算成本。理论分析证明,FedORA与从头训练模型的差异有界,确保遗忘有效性。在表格与图像数据集上的实验表明,FedORA在保持性能的同时,实现与重训练相当的遗忘效果,且计算与通信开销更少。
原文摘要 · Abstract (English)
Federated unlearning has become an attractive approach to address privacy concerns in collaborative machine learning, for situations when sensitive data is remembered by AI models during the machine learning process. It enables the removal of specific data influences from trained models, aligning with the growing emphasis on the "right to be forgotten." While extensively studied in horizontal federated learning, unlearning in vertical federated learning (VFL) remains challenging due to the distributed feature architecture. VFL unlearning includes sample unlearning that removes specific data points' influence and label unlearning that removes entire classes. Since different parties hold complementary features of the same samples, unlearning tasks require cross-party coordination, creating computational overhead and complexities from feature interdependencies. To address such challenges, we propose FedORA (Federated Optimization for data Removal via primal-dual Algorithm), designed for sample and label unlearning in VFL. FedORA formulates the removal of certain samples or labels as a constrained optimization problem solved using a primal-dual framework. Our approach introduces a new unlearning loss function that promotes classification uncertainty rather than misclassification. An adaptive step size enhances stability, while an asymmetric batch design, considering the prior influence of the remaining data on the model, handles unlearning and retained data differently to efficiently reduce computational costs. We provide theoretical analysis proving that the model difference between FedORA and Train-from-scratch is bounded, establishing guarantees for unlearning effectiveness. Experiments on tabular and image datasets demonstrate that FedORA achieves unlearning effectiveness and utility preservation comparable to Retrain with reduced computation and communication overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。