按数据价值差异删数据,模型性能更稳更准
Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning
- 根据数据对模型的贡献值加权,区别对待不同数据点
- 在真实和合成数据集上,预测准确率显著优于传统方法
- 适用于多种学习算法,尤其适合需要隐私保护的场景
随着全球范围内“被遗忘权”立法推进,机器遗忘机制应运而生,用于高效更新模型以实现数据删除并保护用户隐私。然而,现有遗忘算法常忽略一个事实:不同数据点对模型性能的贡献存在差异(即数据价值异质性)。若在遗忘过程中同等处理这些数据,可能损害模型更新后的性能。为此,我们提出数据价值加权遗忘(DVWU)框架,将数据价值异质性融入遗忘过程。具体地,设计基于数据价值的加权策略,并将其集成到遗忘流程中,实现对模型有用性不同的数据点进行差异化遗忘。该框架可广泛适配多种现有遗忘方法。以一步牛顿更新为例,我们开发了输出扰动与目标扰动两种算法,实现认证式遗忘。在合成与真实数据集上的实验表明,本方法在预测性能和鲁棒性方面均优于传统遗忘方法。此外,我们将该加权策略应用于梯度上升法,通过在梯度项中引入权重,验证了其在更广泛的基于梯度的深度遗忘方法中的可扩展性。
原文摘要 · Abstract (English)
As the right to be forgotten becomes legislated worldwide, machine unlearning mechanisms have emerged to efficiently update models for data deletion and enhance user privacy protection. However, existing machine unlearning algorithms frequently neglect the fact that different data points may contribute unequally to model performance (i.e., heterogeneous data values). Treat them equally in machine unlearning procedure can potentially degrading the performance of updated models. To address this limitation, we propose Data Value-Weighted Unlearning (DVWU), a general unlearning framework that accounts for data value heterogeneity into the unlearning process. Specifically, we design a weighting strategy based on data values, which are then integrated into the unlearning procedure to enable differentiated unlearning for data points with varying utility to the model. The DVWU framework can be broadly adapted to various existing machine unlearning methods. We use the one-step Newton update as an example for implementation, developing both output and objective perturbation algorithms to achieve certified unlearning. Experiments on both synthetic and real-world datasets demonstrate that our methods achieve superior predictive performance and robustness compared to conventional unlearning approaches. We further show the extensibility of our framework on gradient ascent method by incorporating the proposed weighting strategy into the gradient terms, highlighting the adaptability of DVWU for broader gradient-based deep unlearning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。