在低维特征空间中实现高效数据遗忘,避免重载数据且更新更快。
Machine Unlearning in Low-Dimensional Feature Subspace
- 基于主成分投影,在低维特征空间分离需保留与需遗忘数据。
- 仅需一次提取特征,计算开销小,遗忘效果优于主流方法。
- 适合需要快速安全删除数据的场景,如隐私保护和模型合规。
机器遗忘(Machine Unlearning, MU)旨在移除特定数据对预训练模型的影响,同时保持对剩余数据的性能。本文提出一种新视角:在低维特征子空间中进行遗忘,该空间具有分离保留数据与遗忘数据的潜力。基于此,我们提出LOFT方法——通过从预训练模型中提取主成分投影,构建一个低维特征子空间,优化投影以最大化保留数据信息、最小化遗忘数据信息。训练时,仅需灵活插入一个小尺寸投影矩阵,并一次性获取预训练骨干网络的特征,无需反复访问原始数据。因此,LOFT缓解了主流方法中的两大问题:大规模数据重载带来的隐私泄露风险,以及对整个预训练模型更新效率低下。大量实验验证了LOFT在多种模型、数据集、任务和应用中均具备显著更低的计算开销和更优的遗忘性能。代码已匿名公开于 https://anonymous.4open.science/r/4352/。
原文摘要 · Abstract (English)
Machine Unlearning (MU) aims at removing the influence of specific data from a pretrained model while preserving performance on the remaining data. In this work, a novel perspective for MU is presented upon low-dimensional feature subspaces, which gives rise to the potentials of separating the remaining and forgetting data herein. This separability motivates our LOFT, a method that proceeds unlearning in a LOw-dimensional FeaTure subspace from the pretrained model skithrough principal projections, which are optimized to maximally capture the information of the remaining data and meanwhile diminish that of the forgetting data. In training, LOFT simply optimizes a small-size projection matrix flexibly plugged into the pretrained model, and only requires one-shot feature fetching from the pretrained backbone instead of repetitively accessing the raw data. Hence, LOFT mitigates two critical issues in mainstream MU methods, i.e., the privacy leakage risk from massive data reload and the inefficiency of updates to the entire pretrained model. Extensive experiments validate the significantly lower computational overhead and superior unlearning performance of LOFT across diverse models, datasets, tasks, and applications. Code is anonymously available at https://anonymous.4open.science/r/4352/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。