提出多目标方法，让大模型高效删除不良内容而不丢掉有用能力
Multi-Objective Large Language Model Unlearning
- 将删减不良数据转化为多目标优化问题，用改进损失函数防梯度爆炸
- 在不重训的前提下，删不良内容效果更好，且保留模型原有性能
- 适合需要快速清除敏感信息又不想重训练的大模型应用
大语言模型（LLM）的机器遗忘近年来备受关注，旨在在不从头重新训练的情况下有效消除模型的不良行为。本文研究了基于梯度上升（GA）的遗忘方法，该方法通过降低模型对目标数据的预测概率来消除其影响。我们分析了两个使该过程不可行的挑战：梯度爆炸和灾难性遗忘。为此，我们提出了多目标大语言模型遗忘（MOLLM）算法。首先将LLM遗忘建模为多目标优化问题，将交叉熵损失修改为遗忘版本以缓解梯度爆炸问题。随后计算一个共同下降方向，使模型能遗忘目标数据的同时保持原有功能。实验结果表明，与现有基于GA的先进方法相比，MOLLM在遗忘效果和模型实用性保持方面均表现更优。源代码已公开于 https://github.com/zibinpan/MOLLM。
原文摘要 · Abstract (English)
Machine unlearning in the domain of large language models (LLMs) has attracted great attention recently, which aims to effectively eliminate undesirable behaviors from LLMs without full retraining from scratch. In this paper, we explore the Gradient Ascent (GA) approach in LLM unlearning, which is a proactive way to decrease the prediction probability of the model on the target data in order to remove their influence. We analyze two challenges that render the process impractical: gradient explosion and catastrophic forgetting. To address these issues, we propose Multi-Objective Large Language Model Unlearning (MOLLM) algorithm. We first formulate LLM unlearning as a multi-objective optimization problem, in which the cross-entropy loss is modified to the unlearning version to overcome the gradient explosion issue. A common descent update direction is then calculated, which enables the model to forget the target data while preserving the utility of the LLM. Our empirical results verify that MoLLM outperforms the SOTA GA-based LLM unlearning methods in terms of unlearning effect and model utility preservation. The source code is available at https://github.com/zibinpan/MOLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。