通过权重重要性引导对比学习,高效删除模型中特定数据的影响。
WSS-CL: Weight Saliency Soft-Guided Contrastive Learning for Efficient Machine Unlearning Image Classification
- 基于权重重要性聚焦关键参数,分两阶段实现精准遗忘。
- 在图像分类任务中性能接近完全遗忘,且模型损失极小。
- 适合需要高效数据删除的工业级模型更新场景。
机器遗忘旨在高效消除特定数据对已训练模型的影响,当前以数据为中心或基于权重的方案常面临遗忘精度不足、稳定性差及跨领域适用性弱的问题。本文提出一种基于权重重要性的两阶段高效机器遗忘方法(WSS-CL),显著缩小与‘精确遗忘’的性能差距。第一阶段在输出空间最大化预测逻辑斯蒂与聚合伪标签间的KL散度,实现高效遗忘;第二阶段采用自监督对比学习,利用缩放特征表示,在特征空间中最大化遗忘样本与保留样本间的距离,其中遗忘样本与其增强对作为正例,保留样本作为负例。实验表明,该方法在保持极低性能损失的前提下,显著提升遗忘效果,适用于监督与自监督场景。
原文摘要 · Abstract (English)
Machine unlearning, the efficient deletion of the impact of specific data in a trained model, remains a challenging problem. Current machine unlearning approaches that focus primarily on data-centric or weight-based strategies frequently encounter challenges in achieving precise unlearning, maintaining stability, and ensuring applicability across diverse domains. In this work, we introduce a new two-phase efficient machine unlearning method for image classification, in terms of weight saliency, leveraging weight saliency to focus the unlearning process on critical model parameters. Our method is called weight saliency soft-guided contrastive learning for efficient machine unlearning image classification (WSS-CL), which significantly narrows the performance gap with "exact" unlearning. First, the forgetting stage maximizes kullback-leibler divergence between output logits and aggregated pseudo-labels for efficient forgetting in logit space. Next, the adversarial fine-tuning stage introduces contrastive learning in a self-supervised manner. By using scaled feature representations, it maximizes the distance between the forgotten and retained data samples in the feature space, with the forgotten and the paired augmented samples acting as positive pairs, while the retained samples act as negative pairs in the contrastive loss computation. Experimental evaluations reveal that our proposed method yields much-improved unlearning efficacy with negligible performance loss compared to state-of-the-art approaches, indicative of its usability in supervised and self-supervised settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。