提出可直接遗忘的视觉变换模型,无需微调即可清除特定数据记忆。
NOVO: Unlearning-Compliant Vision Transformers
- 训练时模拟遗忘,通过分离数据为忘却集与保留集来优化模型
- 遗忘仅需撤回键值,避免性能下降且支持即时操作
- 适合需要快速合规删除用户数据的AI系统
机器遗忘(MUL)旨在使预训练模型在保留其余数据性能的前提下,选择性地遗忘某些训练样本或类别。现有方法依赖于使用遗忘集和/或保留集进行微调,成本高且常导致性能下降。我们提出 { ame},一种面向遗忘的视觉变换器架构,可在不微调请求数据集的情况下直接执行未来遗忘请求。该模型在训练过程中通过模拟遗忘:在每个小批量中随机将类别或子类别分为两个互斥集合——代理遗忘集与保留集,并优化模型使其无法预测遗忘集。遗忘通过撤回键值实现,支持即时遗忘并避免性能退化。模型联合训练可学习键值与原始权重,确保键值撤回不可逆,经成员推断攻击验证有效。在多种数据集、架构和分辨率下的大量实验表明,{ ame} 在无需微调和依赖微调的方法中均表现更优。
原文摘要 · Abstract (English)
Machine unlearning (MUL) refers to the problem of making a pre-trained model selectively forget some training instances or class(es) while retaining performance on the remaining dataset. Existing MUL research involves fine-tuning using a forget and/or retain set, making it expensive and/or impractical, and often causing performance degradation in the unlearned model. We introduce {\pname}, an unlearning-aware vision transformer-based architecture that can directly perform unlearning for future unlearning requests without any fine-tuning over the requested set. The proposed model is trained by simulating unlearning during the training process itself. It involves randomly separating class(es)/sub-class(es) present in each mini-batch into two disjoint sets: a proxy forget-set and a retain-set, and the model is optimized so that it is unable to predict the forget-set. Forgetting is achieved by withdrawing keys, making unlearning on-the-fly and avoiding performance degradation. The model is trained jointly with learnable keys and original weights, ensuring withholding a key irreversibly erases information, validated by membership inference attack scores. Extensive experiments on various datasets, architectures, and resolutions confirm {\pname}'s superiority over both fine-tuning-free and fine-tuning-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。