提出分层联邦遗忘机制，实现大模型安全高效知识清除。
Hierarchical Federated Unlearning for Large Language Models
- 通过任务特定适配器解耦遗忘与保留，降低干扰
- 在多个数据集上实现强保留性能，优于基线方法
- 适合需要持续清除敏感知识的分布式应用
大语言模型日益应用于实际场景，引发隐私与安全担忧，需清除不良知识。机器遗忘虽具前景，但面临两个挑战：（1）遗忘需求常为持续且异构，（2）数据分散且访问不对称。这导致域间与域内干扰，加剧遗忘与保留之间的失衡。为此，我们提出一种可扩展且隐私保护的联邦遗忘方法。该方法通过任务特定适配器学习解耦遗忘与保留，并采用分层合并策略缓解目标冲突，实现稳健、可适应的遗忘更新。在WMDP、MUSE和TOFU基准上的全面实验表明，本方法能有效处理异构遗忘请求，同时相比基线方法保持更强的LLM性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly integrated into real-world applications, raising concerns about privacy, security and the need to remove undesirable knowledge. Machine Unlearning has emerged as a promising solution, yet faces two key challenges: (1) practical unlearning needs are often continuous and heterogeneous, and (2) they involve decentralized, sensitive data with asymmetric access. These factors result in inter-domain and intra-domain interference, which further amplifies the dilemma of unbalanced forgetting and retaining performance. In response, we propose a federated unlearning approach for LLMs that is scalable and privacy preserving. Our method decouples unlearning and retention via task-specific adapter learning and employs a hierarchical merging strategy to mitigate conflicting objectives and enables robust, adaptable unlearning updates. Comprehensive experiments on benchmarks of WMDP, MUSE, and TOFU showed that our approach effectively handles heterogeneous unlearning requests while maintaining strong LLM utility compared with baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。