用神经网络模拟遗忘机制,实现精准删除训练数据而不影响其他性能。
Machine Unlearning using Forgetting Neural Networks
- 基于神经科学的乘性衰减结构,让网络主动遗忘特定数据。
- 在MNIST和Fashion-MNIST上验证,删掉指定数据后模型性能基本不变。
- 可有效抵御成员推理攻击,适合隐私敏感场景下的模型净化。
现代计算机系统存储大量个人数据,虽推动了人工智能与机器学习发展,却带来隐私泄露风险。为保护用户隐私,有时需让训练好的模型忘记部分原始数据。本文提出一种基于遗忘神经网络(FNNs)的新颖卸载方法,该架构受神经科学启发,通过乘性衰减因子显式建模遗忘过程。尽管此前FNNs仅作为理论构想,本文首次实现其具体应用,并在MNIST与Fashion-MNIST基准测试中验证其有效性。我们设计多种带逐神经元遗忘因子的变体,包括依据激活水平进行秩分配的策略。实验表明,该方法能系统性移除与被遗忘数据相关的信息,同时保持对保留数据的良好性能。成员推理攻击结果进一步证实,基于FNN的卸载能有效消除模型中关于训练数据的记忆。这些成果确立了FNNs作为高效且可解释卸载基础的潜力。
原文摘要 · Abstract (English)
Modern computer systems store vast amounts of personal data, enabling advances in AI and ML but risking user privacy and trust. For privacy reasons, it is sometimes desired for an ML model to forget part of the data it was trained on. In this paper, we introduce a novel unlearning approach based on Forgetting Neural Networks (FNNs), a neuroscience-inspired architecture that explicitly encodes forgetting through multiplicative decay factors. While FNNs had previously been studied as a theoretical construct, we provide the first concrete implementation and demonstrate their effectiveness for targeted unlearning. We propose several variants with per-neuron forgetting factors, including rank-based assignments guided by activation levels, and evaluate them on MNIST and Fashion-MNIST benchmarks. Our method systematically removes information associated with forget sets while preserving performance on retained data. Membership inference attacks confirm the effectiveness of FNN-based unlearning in erasing information about the training data from the neural network. These results establish FNNs as a promising foundation for efficient and interpretable unlearning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。