arXiv:2607.09236cs.LG2026-07

提出新评估框架与训练数据,解决大模型删知识时漏删和误删问题。

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

论文配图:Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem
图 1 · 摘自论文原文
  • 设计覆盖多种问法的评测协议,检验删除知识是否真正消失。
  • 在三个大模型上验证,新方法实现最佳删留平衡,性能显著提升。
  • 适合关注隐私保护与模型安全的研究者和开发者使用。

大模型机器遗忘旨在精准移除特定知识,同时保留其他能力,对隐私与安全至关重要。然而现有基准存在严重缺陷:无法检测通过改写或间接提问重现的知识(称为‘欠遗忘’),且缺乏语义、句法、词汇探针来验证无关知识是否完好保留(称为‘过遗忘’)。这两种问题本质上是不对称泛化难题。遗忘评估需涵盖同一事实的多样化表述形式,以测试遗忘效果是否超越原始训练提示;保留评估则需探测远超目标知识的隐式集合——即所有不相关的事实。该保留集定义了有效遗忘范围,但当前数据集缺乏对此边界细粒度标注。为此,我们提出SUITE评估协议与训练语料库,捕捉真实事实领域中的遗忘-保留结构。基于此,训练出的方法性能显著提升,表明训练数据与算法设计同等重要。在此基础上,我们提出JensUn++遗忘算法,在三种大模型的顺序与联合遗忘场景中均实现最优遗忘-保留权衡。代码与数据集已公开。

原文摘要 · Abstract (English)

Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under-forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over-forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine-grained annotation of this forget-retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget-retain structure for real-world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget-retain utility trade-off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at https://amitpeleg.github.io/forget-narrowly-retain-broadly

大模型遗忘隐私评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。