让大模型自动识别该删哪些词,实现精准无监督遗忘。
Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance

- 用保留任务冲突度判断每个词是否该删,无需额外标注。
- 在TOFU和RWKU数据集上超越现有方法,遗忘更彻底且不伤通用能力。
- 轻量级框架,直接从模型内部表示学词的重要性,适合实际部署。
机器遗忘旨在移除训练好的模型中特定知识,同时保持其通用能力。对于自回归语言模型,遗忘样本中的不同词对遗忘的贡献并不相同。现有方法或忽略这种差异性,或依赖辅助模型、启发式规则或外部标注来估计词的重要性。本文提出通过保留目标的交互来刻画这一差异:一个词若在最小化遗忘损失时不影响保留最优性,则其为特定于遗忘的。我们将其形式化为联合优化问题,在自然分离条件下可恢复理想遗忘词集。基于此,提出交替词权重遗忘(ATWU)框架,使用隐藏状态上的简单线性评分器联合学习词的遗忘重要性与模型参数,无需外部词级监督。在TOFU和RWKU数据集上,ATWU实现当前最佳遗忘-保留平衡,优于样本级方法、概率加权启发式及辅助模型方法。此外,学习到的得分与真实遗忘片段高度一致,表明其能捕捉语义上有意义的词级遗忘信号。结果表明,保留冲突是识别应遗忘内容的有效标准,支持从模型表示中无监督地学习词级遗忘重要性,计算开销极小。
原文摘要 · Abstract (English)
Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities. For autoregressive language models, not all tokens in a forget sample are equally relevant to forgetting. Existing approaches either ignore this heterogeneity or rely on auxiliary models, heuristics, or external annotations to estimate each token's relevance for forgetting. We instead characterize it through the interaction with the retain objective: a token is forget-specific to the extent that minimizing the forget loss on that token does not conflict with retain optimality. We formalize this perspective as a joint optimization problem over the model parameters and the token weights and show that, under a natural separation condition, the resulting objective recovers the oracle forget-specific token support. Motivated by this formulation, we introduce Alternating Token-Weighted Unlearning (ATWU), a lightweight framework that jointly learns token forget-specificity and model parameters during unlearning using a simple linear scorer over the hidden states, without external token level supervision. Across TOFU and RWKU, ATWU achieves state of the art forget-retain trade-offs, outperforming sample-level methods, probability-based token weighting heuristics, and auxiliary-model-based approaches. Moreover, the learned scores align substantially better with ground truth forget-specific spans, indicating that ATWU identifies semantically meaningful token level forgetting signals. Overall, our results suggest that retain conflict provides an effective criterion for identifying what language models should forget, enabling unsupervised learning of token level forget-specificity directly from model representations with minimal computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。