通过可逆的稀疏适配器实现精准遗忘,降低联邦学习中的遗忘成本。
Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter
- 按层敏感度筛选关键层,用稀疏适配器定向覆盖旧知识。
- 遗忘后可恢复原模型,且代价远低于重训练。
- 适用于需精准删除隐私或有害信息的联邦学习场景。
联邦学习是一种保护隐私的协同模型训练范式。实际应用中,不仅需要持续学习新知识,还需保障旧知识被遗忘的权利(即联邦遗忘),尤其对敏感信息或有害知识尤为重要。然而,现有方法面临盲目遗忘跨客户端知识、遗忘不可逆及成本高等挑战。为此,我们提出FUSED方法:首先通过分析各层对知识的敏感度识别关键层,并为敏感层构建稀疏遗忘适配器;随后在不修改原始参数的前提下训练适配器,以剩余知识覆盖遗忘知识。该知识覆盖机制有效缓解了盲目遗忘问题。此外,独立适配器的设计使遗忘可逆,并显著降低遗忘开销。在三个数据集上多种遗忘场景的实验表明,FUSED性能接近重训练,优于所有基线方法,且大幅降低遗忘成本。
原文摘要 · Abstract (English)
Federated Learning is a promising paradigm for privacy-preserving collaborative model training. In practice, it is essential not only to continuously train the model to acquire new knowledge but also to guarantee old knowledge the right to be forgotten (i.e., federated unlearning), especially for privacy-sensitive information or harmful knowledge. However, current federated unlearning methods face several challenges, including indiscriminate unlearning of cross-client knowledge, irreversibility of unlearning, and significant unlearning costs. To this end, we propose a method named FUSED, which first identifies critical layers by analyzing each layer's sensitivity to knowledge and constructs sparse unlearning adapters for sensitive ones. Then, the adapters are trained without altering the original parameters, overwriting the unlearning knowledge with the remaining knowledge. This knowledge overwriting process enables FUSED to mitigate the effects of indiscriminate unlearning. Moreover, the introduction of independent adapters makes unlearning reversible and significantly reduces the unlearning costs. Finally, extensive experiments on three datasets across various unlearning scenarios demonstrate that FUSED's effectiveness is comparable to Retraining, surpassing all other baselines while greatly reducing unlearning costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。