让大模型快速删掉敏感信息，不伤其他知识。
ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
- 用模型编辑实现精准知识重映射，不重新训练。
- 仅需少量样本即可删除敏感内容，保持模型整体能力。
- 适合需要快速清理隐私数据的AI应用开发人员。
大语言模型因在海量网络语料上训练，可能保留敏感信息（即可能引发有害生成的输入），引发隐私与安全担忧。现有机器遗忘方法多依赖重训练或激进微调，要么计算成本高，要么易损害相关知识与模型整体性能。本文将机器遗忘重新定义为精确的知识重映射问题，提出零样本遗忘框架 ZeroUnlearn。该方法通过模型编辑，将敏感输入映射至中性目标状态并移除其原始表征。借助乘法参数更新与闭式解，强制表征正交性，实现高效且精准的遗忘。我们进一步扩展出基于梯度的变体，支持多样本遗忘。实验表明，该方法优于现有基线，同时保持模型通用性能。代码已开源：https://github.com/XMUDeepLIT/ZeroUnlearn。
原文摘要 · Abstract (English)
Large language models inevitably retain sensitive information, defined as inputs that may induce harmful generations, due to training on massive web corpora, raising concerns for privacy and safety. Existing machine unlearning methods primarily rely on retraining or aggressive fine-tuning, which are either computationally expensive or prone to degrading related knowledge and overall model utility. In this work, we reformulate machine unlearning as a precise knowledge re-mapping problem via model editing. We propose ZeroUnlearn, a few-shot unlearning framework. It overwrites sensitive inputs by mapping them to a neutral target state and removing their original representations. ZeroUnlearn enforces representational orthogonality through a multiplicative parameter update with a closed-form solution, enabling efficient and targeted unlearning. We further extend ZeroUnlearn to a gradient-based variant for multi-sample unlearning. Experiments demonstrate that our approach outperforms existing baselines while preserving general model utility. Our code is available at the github: https://github.com/XMUDeepLIT/ZeroUnlearn.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。