无需重训练，高效删除大模型过时信息并更新知识
LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models
- 三目标优化：反向梯度删旧知，正向梯度学新知，KL散度保原貌
- 在未更新数据上准确率提升20%，过时信息显著遗忘，原有能力保持
- 适用于需快速修正模型错误或更新知识的场景，如医疗、法律等
大型语言模型虽在多个领域带来革新，但其预训练中嵌入的过时或问题性知识带来挑战。本文提出LLM Surgery框架，通过优化三元目标函数，在不从头训练的前提下高效修改模型行为：对需删除的知识集执行反向梯度，对新增知识集执行正向梯度，对保留文本子集最小化KL散度，确保修改前后输出一致性。由于缺乏针对该任务的公开数据集，我们构建了新数据集与评估基准。基于Llama2-7B实验表明，该方法在删除集上实现显著遗忘，更新集准确率提升20%，且在保留集上性能不受影响。
原文摘要 · Abstract (English)
Large language models (LLMs) have revolutionized various domains, yet their utility comes with significant challenges related to outdated or problematic knowledge embedded during pretraining. This paper addresses the challenge of modifying LLMs to unlearn problematic and outdated information while efficiently integrating new knowledge without retraining from scratch. Here, we propose LLM Surgery, a framework to efficiently modify LLM behaviour by optimizing a three component objective function that: (1) Performs reverse gradient on unlearning dataset (problematic and outdated information), (2) Performs gradient descent on the update dataset (new and updated information), and (3) Minimizes the KL divergence on the retain dataset (small subset of unchanged text), ensuring alignment between pretrained and modified model outputs. Due to the lack of publicly available datasets specifically tailored for our novel task, we compiled a new dataset and an evaluation benchmark. Using Llama2-7B, we demonstrate that LLM Surgery can achieve significant forgetting on the unlearn set, a 20\% increase in accuracy on the update set, and maintain performance on the retain set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。