arXiv:2507.23257cs.LGcs.AI2025-07

提出新方法让模型高效删数据,速度比现有技术快得多。

Efficient Machine Unlearning via Influence Approximation

  • 从增量学习角度近似计算样本影响,避开昂贵的海塞矩阵运算。
  • 在多个数据集和模型上验证,删除效果好且推理效率提升10倍以上。
  • 适合需要频繁删数据的场景,如医疗、金融等隐私敏感领域。

随着隐私问题日益突出,机器遗忘(machine unlearning)——即让模型“忘记”特定训练数据——受到越来越多关注。现有方法中,基于影响估计的遗忘机制因无需重新训练而成为主流,但其需对所有训练样本和参数计算海塞矩阵及其逆矩阵,带来巨大计算开销,难以应用于大规模模型或高频删除请求场景。本文受认知科学启发,发现记忆(增量学习)比遗忘(机器遗忘)更高效,并建立两者之间的理论联系。由此,我们提出从增量学习视角出发的影响力近似遗忘(IAU)算法,避免了传统方法中耗时的海塞计算。大量实证评估表明,IAU在删除保证、遗忘效率与模型性能之间取得更好平衡,显著优于当前最先进方法,适用于多种数据集与模型架构。代码已开源。

原文摘要 · Abstract (English)

Due to growing privacy concerns, machine unlearning, which aims at enabling machine learning models to ``forget" specific training data, has received increasing attention. Among existing methods, influence-based unlearning has emerged as a prominent approach due to its ability to estimate the impact of individual training samples on model parameters without retraining. However, this approach suffers from prohibitive computational overhead arising from the necessity to compute the Hessian matrix and its inverse across all training samples and parameters, rendering it impractical for large-scale models and scenarios involving frequent data deletion requests. This highlights the difficulty of forgetting. Inspired by cognitive science, which suggests that memorizing is easier than forgetting, this paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning). This connection allows machine unlearning to be addressed from the perspective of incremental learning. Unlike the time-consuming Hessian computations in unlearning (forgetting), incremental learning (memorizing) typically relies on more efficient gradient optimization, which supports the aforementioned cognitive theory. Based on this connection, we introduce the Influence Approximation Unlearning (IAU) algorithm for efficient machine unlearning from the incremental perspective. Extensive empirical evaluations demonstrate that IAU achieves a superior balance among removal guarantee, unlearning efficiency, and comparable model utility, while outperforming state-of-the-art methods across diverse datasets and model architectures. Our code is available at https://github.com/Lolo1222/IAU.

机器遗忘高效算法隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。