通过消除模型对遗忘数据的归属感，实现更安全的LLM删忆。
De-attribute to Forget for LLM Unlearning
- 用归因分数下降替代损失最大化，指导模型删忆。
- 在保持模型性能的同时，有效降低遗忘数据的关联性。
- 适合需要合规删忆且兼顾实用性的AI系统应用。
大型语言模型（LLMs）的快速发展引发了训练数据不当使用的问题，推动了模型删忆研究的兴起。现有方法多依赖优化预测损失（如最大化遗忘集上的损失），但常面临过度遗忘和模型效用下降等问题。本文首次将删忆目标重构为使数据归因归零，提出基于数据归因奖励的强化学习框架DareU。该方法通过减少生成结果对遗忘数据所有者的归因得分（即去归因），更新模型。实验采用分类器近似归因，验证DareU在保持模型实用性的同时，显著提升删忆效果，优于现有基线方法。
原文摘要 · Abstract (English)
The rapid development of large language models (LLMs) has raised concerns on the use of inappropriate data for training, which has led to a growing interest in LLM unlearning. Many existing LLM unlearning approaches rely on optimizing prediction loss(es), such as maximizing the loss on the forget set, but often face critical issues like over-forgetting and poor model utility. To address them, this paper novelly frames the optimization objective for LLM unlearning as one of zeroing out data attribution instead. In particular, we propose the first LLM unlearning framework based on data attribution rewards called DareU that performs reinforcement learning to update the LLM by reducing the attribution score of its generated responses (i.e., de-attributing) to the forget data owners. Empirical evaluation using an LLM classifier as an efficient approximation of attribution shows that DareU outperforms existing baselines by achieving effective unlearning while balancing forget quality and model utility well.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。