通过构建支持路径图，扩展遗忘范围以更彻底删除语言模型中的敏感信息。
Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds
- 基于图结构建模知识支持路径，动态扩展需删除的范围。
- 在TOFU和PISTOL基准上，漏泄率降低最高达49.5个百分点。
- 适合需要合规删除、隐私保护的企业级大模型管理场景。
企业常对语言模型进行私有数据微调，但后续可能因隐私、合同或合规要求需删除特定知识。选择性遗忘可在保留模型性能的同时移除指定内容，优于全量重训，但现有方法仅以明确标识的遗忘样本为删除范围。当目标知识可通过改写、别名或邻近样本恢复时，此法不足。本文提出GRAPHSU，通过构建加权支持路径图，传播删除压力并为高风险邻居施加渐进式遗忘强度，扩大删除范围。在基于虚构作者问答的TOFU与围绕关联事实样本的PISTOL两个基准上，使用GPT-2 Medium和Llama-3.2-3B-Instruct模型，GRAPHSU在所有删除设置中实现最低可行软漏泄，相比仅依赖种子样本的基线，漏泄率下降最多达49.5个百分点，证明有效的企业级遗忘需控制支持路径，而不仅是遗忘种子。
原文摘要 · Abstract (English)
Enterprises fine-tune language models on proprietary data that may later require removal due to privacy, contractual, or compliance obligations. Selective unlearning removes requested knowledge while preserving model utility, offering a practical alternative to full retraining, but existing methods treat the explicitly identified forget examples as the complete deletion scope. This is insufficient when target knowledge remains recoverable through paraphrases, aliases, or neighboring training examples. We propose GRAPHSU, a graph-guided controller that expands the deletion scope beyond forget seeds by constructing a weighted support-route graph, propagating deletion pressure through it, and applying graded forgetting strengths to high-risk neighbors. On the Task of Fictitious Unlearning (TOFU), a synthetic author-profile question-answering benchmark, and PISTOL, a structural-unlearning benchmark built around interconnected factual samples, with GPT-2 Medium and Llama-3.2-3B-Instruct, GRAPHSU achieves the lowest utility-feasible soft leakage across all deletion settings, reducing leakage by up to 49.5 percentage points over a matched seed-only baseline, demonstrating that effective enterprise unlearning requires controlling support routes, not just forget seeds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。