用模型自信念对抗记忆残留,让大模型真正遗忘敏感内容
LLM Unlearning with LLM Beliefs
- 通过模型高置信度生成物构建反向约束,直接抑制概率挤出效应
- 在多个基准上实现90%以上目标响应消除,且不降低整体生成质量
- 适合需要强数据安全性的应用,如医疗、金融对话系统
大规模语言模型在海量语料上训练,可能记住敏感或有害内容,后续输出中重现。现有遗忘方法多依赖梯度上升降低特定响应概率,但会引发关键副作用:概率质量被重分配至高可能性区域,常表现为与目标语义相关的改写版本。我们称之为‘挤压效应’,这解释了为何许多方法仅产生虚假遗忘,且被自动指标(如ROUGE、真值率)误判为成功。为此,我们提出一种基于自信念(model beliefs)的自举(BS)框架,将挤压效应与模型自身高置信生成关联起来。由于模型信念天然捕获了概率挤出的高可能性区域,将其纳入遗忘目标可直接对抗该效应。通过联合抑制目标响应与模型信念,BS-T(token级)削弱高概率词元,而BS-S(序列级)移除整个高置信生成,共同实现更彻底遗忘的同时保持模型实用性。跨多种模型家族和基准的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Large language models trained on vast corpora inherently risk memorizing sensitive or harmful content, which may later resurface in their outputs. Prevailing unlearning methods generally rely on gradient ascent and its variants to lower the probability of specific target responses. However, we find that this strategy induces a critical side effect: probability mass is redistributed into high-likelihood regions, often corresponding to semantically related rephrasings of the targets. We refer to this as the squeezing effect, which explains why many methods yield merely spurious unlearning, a problem further obscured by automated metrics (e.g., ROUGE, truth ratio) that misreport actual success. To address this, we propose a bootstrapping (BS) framework that explicitly links the squeezing effect with the model's own high-confidence generations, namely its model beliefs. Since model beliefs inherently capture the very high-likelihood regions where probability mass is squeezed, incorporating them into the unlearning objective directly counters the squeezing effect. By jointly suppressing both target responses and model beliefs, BS-T (token) attenuates high-probability tokens, whereas BS-S (sequence) removes entire high-confidence generations, together achieving more thorough forgetting while preserving utility. Extensive experiments across diverse benchmarks with various model families confirm the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。