通过过滤无效遗忘数据,显著提升模型删数效率。
FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning
- 基于数据相似性识别可忽略的删除项,减少无意义计算。
- 实验显示删数请求缩小超70%,执行时间大幅降低。
- 适合需频繁处理隐私删除请求的系统开发者。
机器遗忘是新兴领域,旨在从已训练模型中选择性移除特定数据样本,以应对隐私问题、满足数据保护法规并修正错误或偏见。与传统静态模型不同,机器遗忘支持动态更新,使模型可在不完全重训的前提下实现“遗忘”。现有方法在删除请求较少时更高效,但仍有优化空间。本文提出“非必要遗忘”概念:某些数据移除后模型表现无明显变化。为此,我们设计FUNU方法,通过检测删除请求中的数据点是否在剩余数据集中有相似邻居,来识别可忽略的删除项。该方法利用参考模型设定邻居搜索参数,受模型记忆现象启发。我们提供了FUNU的隐私保障理论分析,并通过大量实验验证其有效性,结果表明其能显著减少删除请求规模并缩短执行时间。
原文摘要 · Abstract (English)
Machine unlearning is an emerging field that selectively removes specific data samples from a trained model. This capability is crucial for addressing privacy concerns, complying with data protection regulations, and correcting errors or biases introduced by certain data. Unlike traditional machine learning, where models are typically static once trained, machine unlearning facilitates dynamic updates that enable the model to ``forget'' information without requiring complete retraining from scratch. There are various machine unlearning methods, some of which are more time-efficient when data removal requests are fewer. To decrease the execution time of such machine unlearning methods, we aim to reduce the size of data removal requests based on the fundamental assumption that the removal of certain data would not result in a distinguishable retrained model. We first propose the concept of unnecessary unlearning, which indicates that the model would not alter noticeably after removing some data points. Subsequently, we review existing solutions that can be used to solve our problem. We highlight their limitations in adaptability to different unlearning scenarios and their reliance on manually selected parameters. We consequently put forward FUNU, a method to identify data points that lead to unnecessary unlearning. FUNU circumvents the limitations of existing solutions. The idea is to discover data points within the removal requests that have similar neighbors in the remaining dataset. We utilize a reference model to set parameters for finding neighbors, inspired from the area of model memorization. We provide a theoretical analysis of the privacy guarantee offered by FUNU and conduct extensive experiments to validate its efficacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。