arXiv:2603.12275cs.CLcs.LG2026-03被引 1

提出GONE框架与NEDS方法,实现对知识图谱中结构化知识的精准删去。

GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

  • 基于知识图谱设计新评估基准,分离删除、推理泄露和遗忘三类影响。
  • 在LLaMA-3-8B和Mistral-7B上实现0.839的局部性与1.000的删除有效性。
  • 适合关注模型隐私与知识可控性的研究者与开发者使用。

大语言模型因具备大规模记忆和消化训练数据的能力,导致安全、隐私和知识产权问题日益突出。现有方法多聚焦于平铺的句子级数据,忽视了自然结构数据中的关系、多跳推理与复杂知识。为此,本文提出图谱遗忘与节点擦除(GONE)基准,用于评估大语言模型在知识图谱事实上的知识删去能力。该基准可分离三类删去效应:直接事实删除、基于推理的泄露和灾难性遗忘。同时,提出邻域扩展分布塑造(NEDS)新框架,利用图连通性识别相关邻居,建立被遗忘事实与其语义邻域间的精确决策边界。在LLaMA-3-8B和Mistral-7B上,多种编辑与删去方法的评估表明,NEDS在GONE及其他基准上表现优异,删去有效性达1.000,局部性为0.839。代码已公开于https://anonymous.4open.science/r/GONE-4679/。

原文摘要 · Abstract (English)

Unlearning knowledge is a pressing and challenging task in Large Language Models (LLMs) because of their unprecedented capability to memorize and digest training data at scale, raising more significant issues regarding safety, privacy, and intellectual property. However, existing works, including parameter editing, fine-tuning, and distillation-based methods, are all focused on flat sentence-level data but overlook the relational, multi-hop, and reasoned knowledge in naturally structured data. In response to this gap, this paper introduces Graph Oblivion and Node Erasure (GONE), a benchmark for evaluating knowledge unlearning over structured knowledge graph (KG) facts in LLMs. This KG-based benchmark enables the disentanglement of three effects of unlearning: direct fact removal, reasoning-based leakage, and catastrophic forgetting. In addition, Neighborhood-Expanded Distribution Shaping (NEDS), a novel unlearning framework, is designed to leverage graph connectivity and identify anchor correlated neighbors, enforcing a precise decision boundary between the forgotten fact and its semantic neighborhood. Evaluations on LLaMA-3-8B and Mistral-7B across multiple knowledge editing and unlearning methods showcase NEDS's superior performance (1.000 on unlearning efficacy and 0.839 on locality) on GONE and other benchmarks. Code is available at https://anonymous.4open.science/r/GONE-4679/.

知识图谱模型遗忘大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。