用轻量方法让大模型删掉特定知识，不重训也不丢能力。
Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs
- 结合影响函数与二阶优化，精准移除数据影响。
- 在多类任务中实现有效遗忘，且整体性能稳定。
- 适合需快速删除敏感信息的模型应用落地。
本文介绍LIBU（LoRA增强型基于影响的遗忘）算法，用于解决大语言模型中删除特定知识的问题（SemEval-2025 Task 4：从大语言模型中删除敏感内容）。该方法结合经典的影响函数以消除数据对模型的影响，并采用二阶优化来稳定模型的整体性能。实验表明，这种轻量级方案适用于多种任务场景下的大模型遗忘，无需从头训练即可实现有效知识删除，同时保持模型原有功能不受损。
原文摘要 · Abstract (English)
This paper describes LIBU (LoRA enhanced influence-based unlearning), an algorithm to solve the task of unlearning - removing specific knowledge from a large language model without retraining from scratch and compromising its overall utility (SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models). The algorithm combines classical \textit{influence functions} to remove the influence of the data from the model and \textit{second-order optimization} to stabilize the overall utility. Our experiments show that this lightweight approach is well applicable for unlearning LLMs in different kinds of task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。