通过知识密度定位与层重插入,彻底清除大模型中的有害知识。
Beyond Superficial Forgetting: Thorough Unlearning through Knowledge Density Estimation and Block Re-insertion
- 用知识密度估计定位有害知识集中的模型层
- 通过重插入策略实现有害知识的彻底消除
- 适合需要严格隐私合规的大模型应用场景
机器遗忘旨在不从头训练的情况下,选择性移除预训练模型中的有害知识,对解决大语言模型(LLMs)的隐私、合规与伦理问题至关重要。然而现有方法常难以彻底清除有害知识,残留内容易被恢复。为此,我们提出知识密度引导的块重插入遗忘方法(KUnBR),先通过知识密度估计量化并定位包含最多有害知识的层,再采用层重插入策略,将有害知识丰富的层提取并重新插入原模型,绕过遮蔽层带来的梯度阻碍,确保遗忘过程中的有效梯度传播。在多个遗忘与通用能力基准上的大量实验表明,KUnBR在实现顶尖遗忘性能的同时,保持了模型的可用性。
原文摘要 · Abstract (English)
Machine unlearning, which selectively removes harmful knowledge from a pre-trained model without retraining from scratch, is crucial for addressing privacy, regulatory compliance, and ethical concerns in Large Language Models (LLMs). However, existing unlearning methods often struggle to thoroughly remove harmful knowledge, leaving residual harmful knowledge that can be easily recovered. To address these limitations, we propose Knowledge Density-Guided Unlearning via Blocks Reinsertion (KUnBR), a novel approach that first identifies layers with rich harmful knowledge and then thoroughly eliminates the harmful knowledge via re-insertion strategy. Our method introduces knowledge density estimation to quantify and locate layers containing the most harmful knowledge, enabling precise unlearning. Additionally, we design a layer re-insertion strategy that extracts and re-inserts harmful knowledge-rich layers into the original LLM, bypassing gradient obstruction caused by cover layers and ensuring effective gradient propagation during unlearning. Extensive experiments conducted on several unlearning and general capability benchmarks demonstrate that KUnBR achieves state-of-the-art forgetting performance while maintaining model utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。