用均值教师算法让语言模型遗忘特定内容，同时保持整体能力。
A mean teacher algorithm for unlearning of language models
- 采用均值教师框架，渐进式更新教师模型以减少记忆残留。
- 结合新损失函数NLUL，使模型在遗忘时性能下降更小。
- 适合需要隐私保护或内容可控的模型优化场景。
语言模型去记忆的目标是降低对特定文本实例的过度记忆，同时保留模型的一般能力。尽管已有多种方法，但在不明显损害模型实用性的情况下减少大规模数据的记忆仍具挑战。本文研究了来自持续学习领域的均值教师算法（Tarvainen & Valpola, 2017），该方法通过渐进方式调整教师模型。我们证明，均值教师可近似于一种缓慢的自然梯度下降（NGD）路径，其本身倾向于低曲率更新，从而更少破坏模型性能。虽然慢速自然梯度可能面临梯度消失问题，我们引入了一种新的去记忆损失函数“负对数未似然”（NLUL），有效避免此问题。实验表明，均值教师与NLUL的结合在MUSE基准测试（Shi et al., 2024）中提升了部分指标。
原文摘要 · Abstract (English)
One of the goals of language model unlearning is to reduce memorization of selected text instances while retaining the model's general abilities. Despite various proposed methods, reducing memorization of large datasets without noticeable degradation in model utility remains challenging. In this paper, we investigate the mean teacher algorithm (Tarvainen & Valpola, 2017), a simple proximal optimization method from continual learning literature that gradually modifies the teacher model. We show that the mean teacher can approximate a trajectory of a slow natural gradient descent (NGD), which inherently seeks low-curvature updates that are less likely to degrade the model utility. While slow NGD can suffer from vanishing gradients, we introduce a new unlearning loss called "negative log-unlikelihood" (NLUL) that avoids this problem. We show that the combination of mean teacher and NLUL improves some metrics on the MUSE benchmarks (Shi et al., 2024).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。