提出非线性路径方法,高效清除模型中的特定数据记忆。
Exploring Nonlinear Pathway in Parameter Space for Machine Unlearning
- 利用模式连接性构建非线性遗忘路径,避免权重纠缠
- 通过参数掩码提升遗忘效果并降低计算开销
- 自适应调节惩罚系数,无需人工调参,适合各类遗忘场景
机器遗忘(Machine Unlearning, MU)旨在从已训练模型中移除特定训练数据的信息,以满足隐私法规和用户请求。现有基于任务算术的线性参数更新方法存在权重纠缠问题。本文提出一种新框架Mode Connectivity Unlearning(MCU),利用模式连接性在参数空间中寻找非线性遗忘路径。为提升性能与效率,引入参数掩码策略,增强遗忘效果并减少计算开销;同时提出自适应调整遗忘惩罚系数的方法,动态平衡遗忘质量与预测性能,避免经验性超参数调优。与仅生成单一遗忘模型的传统方法不同,MCU可发现沿路径的一系列遗忘模型。整体上,MCU为即插即用框架,可无缝集成至任意现有MU方法中,持续提升遗忘有效性。图像分类任务上的大量实验表明,MCU表现更优。代码已开源:https://github.com/TIML-Group/Mode-Connectivity-Unlearning。
原文摘要 · Abstract (English)
Machine Unlearning (MU) aims to remove the information of specific training data from a trained model, ensuring compliance with privacy regulations and user requests. While one line of existing MU methods relies on linear parameter updates via task arithmetic, they suffer from weight entanglement. In this work, we propose a novel MU framework called Mode Connectivity Unlearning (MCU) that leverages mode connectivity to find an unlearning pathway in a nonlinear manner. To further enhance performance and efficiency, we introduce a parameter mask strategy that not only improves unlearning effectiveness but also reduces computational overhead. Moreover, we propose an adaptive adjustment strategy for our unlearning penalty coefficient to adaptively balance forgetting quality and predictive performance during training, eliminating the need for empirical hyperparameter tuning. Unlike traditional MU methods that identify only a single unlearning model, MCU uncovers a spectrum of unlearning models along the pathway. Overall, MCU serves as a plug-and-play framework that seamlessly integrates with any existing MU methods, consistently improving unlearning efficacy. Extensive experiments on the image classification task demonstrate that MCU achieves superior performance. The codes are available at https://github.com/TIML-Group/Mode-Connectivity-Unlearning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。