让大模型精准删掉敏感信息，同时不丢原有能力
SIMU: Selective Influence Machine Unlearning
- 只更新记忆敏感信息的关键神经元，减少对整体模型的干扰
- 在遗忘目标信息上效果相当，但原始知识保留率显著提升
- 适合需要安全可控的大模型应用，如医疗、金融领域
大型语言模型（LLMs）对敏感信息的不当记忆引发了安全机制的需求。为此，机器遗忘技术应运而生，使模型能够精确遗忘特定信息。目前基于一阶和二阶优化器的方法在遗忘能力上取得进展，但往往损害模型原有能力，导致遗忘后知识丢失严重。为此，我们提出选择性影响机器遗忘（SIMU），一种两阶段框架，通过仅更新编码遗忘集信息的关键神经元，实现对二阶优化器方法的增强。该策略限制更新范围，使模型在保持相近遗忘效果的同时，显著优于现有方法在保留原始知识方面的表现。
原文摘要 · Abstract (English)
The undesired memorization of sensitive information by Large Language Models (LLMs) has emphasized the need for safety mechanisms that can regulate model behavior. This has led to the development of machine unlearning techniques that enable models to precisely forget sensitive and unwanted information. For machine unlearning, first-order and second-order optimizer-based methods have shown significant progress in enabling LLMs to forget targeted information. However, in doing so, these approaches often compromise the model's original capabilities, resulting in unlearned models that struggle to retain their prior knowledge and overall utility. To address this, we propose Selective Influence Machine Unlearning (SIMU), a two-step framework that enhances second-order optimizer-based unlearning by selectively updating only the critical neurons responsible for encoding the forget-set. By constraining updates to these targeted neurons, SIMU achieves comparable unlearning efficacy while substantially outperforming current methods in retaining the model's original knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。