提出Lethe方法,让联邦学习在删除数据后仍能长期防止知识复活。
\textsc{Lethe}: Principled Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning
- 双流机制:分别处理被删客户端和保留客户端的数据更新
- 实验显示多数现有方法易出现知识复活,而Lethe在长时训练后仍保持低于1%的泄露率
- 适用于图像与文本任务,支持多层级删除需求
联邦去学习(Federated Unlearning, FU)旨在从全局模型中移除特定知识。现有研究通常假设去学习完成后协作终止,忽略了实际部署中保留客户端仍会继续训练的场景。本文识别出一种关键失效模式——知识复活,即仅在保留数据上继续训练数轮即可重新激活已删除的知识。实验证明,许多先进FU方法均存在此问题。为此,我们提出Lethe,一种面向持续知识擦除的新型去学习方法。每轮迭代中,Lethe同时处理来自被删客户端的遗忘流和来自保留客户端的保留流,将更新方向引导至两流反向对齐的区域,阻止保留数据训练回溯至已遗忘的知识空间。结果表明,Lethe在后续联邦训练中展现出更强的去学习持久性。跨多种模型、数据集及不同删除程度的广泛实验验证,Lethe以统一方式支持所有删除层级,在计算机视觉与自然语言处理任务中表现稳定,绝大多数情况下泄露率(RR)始终低于1%,即使在极长的后续训练周期下亦如此。
原文摘要 · Abstract (English)
Federated unlearning (FU) aims to erase knowledge from a global model. Existing studies commonly assume that federated collaboration terminates after unlearning, overlooking a deployment-realistic scenario where training continues on the remaining clients after deletion requests are fulfilled. In this work, we identify a critical failure mode, termed knowledge resurfacing, revealing that continued training on retained data alone can reactivate unlearned knowledge in a few rounds. Empirically, we demonstrate that many state-of-the-art FU methods are prone to knowledge resurfacing. We then propose Lethe, a novel unlearning method for persistent knowledge erasure in federated settings. In each iteration, Lethe operates on a forget stream from the unlearning client and a retain stream from the retained clients. It redirects unlearning updates toward a region where the two streams are anti-aligned, discouraging retained-data training from moving back toward the forgotten knowledge. Consequently, Lethe ensures stronger unlearning persistence during subsequent federated training. Extensive experiments across diverse models, datasets, and unlearning levels validate that Lethe supports all levels of unlearning in a unified manner across both CV and NLP tasks, demonstrating consistently low RR, below 1% in most cases, even after an extremely long horizon of follow-up training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。