提出混合方法,高效准确地删除模型中被撤销的数据痕迹。
A hybrid framework for effective and efficient machine unlearning
- 根据撤销数据量大小自动选择重训练或参数修改策略。
- 在保持高准确率的同时,提升1.5至8倍的卸载效率。
- 适合需要快速响应隐私请求的实际系统部署。
近期提出的机器卸载(MU)旨在移除已训练模型中被撤销样本的影响,以解决用户隐私问题。不同于耗时的从头重新训练,现有研究分为精确卸载和近似卸载两类,各有侧重于精度与效率。本文提出一种新型混合策略,在两者基础上实现整体优化:通过估算撤销带来的重训练开销,若开销轻量则执行重训练以获得与从头训练一致的准确参数;否则直接修改当前参数以提升效率。特别地,为改善后者的精度,我们设计了一种轻量级优化版本,在运行时仅增加少量开销即可修正输出模型。我们深入研究两种方法的边界,实现自适应选择。在真实数据集上的大量实验表明,所提方法可将卸载效率提升1.5×至8×,同时保持相当的精度水平。
原文摘要 · Abstract (English)
Recently machine unlearning (MU) is proposed to remove the imprints of revoked samples from the already trained model parameters, to solve users' privacy concern. Different from the runtime expensive retraining from scratch, there exist two research lines, exact MU and approximate MU with different favorites in terms of accuracy and efficiency. In this paper, we present a novel hybrid strategy on top of them to achieve an overall success. It implements the unlearning operation with an acceptable computation cost, while simultaneously improving the accuracy as much as possible. Specifically, it runs reasonable unlearning techniques by estimating the retraining workloads caused by revocations. If the workload is lightweight, it performs retraining to derive the model parameters consistent with the accurate ones retrained from scratch. Otherwise, it outputs the unlearned model by directly modifying the current parameters, for better efficiency. In particular, to improve the accuracy in the latter case, we propose an optimized version to amend the output model with lightweight runtime penalty. We particularly study the boundary of two approaches in our frameworks to adaptively make the smart selection. Extensive experiments on real datasets validate that our proposals can improve the unlearning efficiency by 1.5$\times$ to 8$\times$ while achieving comparable accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。