arXiv:2509.15621cs.CLcs.LG2025-09被引 2

让大模型删除特定概念,而非单个句子,更精准保护隐私。

Concept Unlearning in Large Language Models via Self-Constructed Knowledge Triplets

  • 用知识图谱表示模型内部知识,以节点和边定义要删的概念。
  • 通过自动生成知识三元组实现概念级遗忘,保留无关知识不受损。
  • 适合需要移除人物、事件等抽象概念的隐私保护场景。

机器遗忘(MU)近年来受到广泛关注,被视为解决大语言模型(LLM)中隐私与版权问题的有效方案。现有方法旨在移除特定目标句子,同时最小化对无关知识的损害,但需明确指定句子,无法支持删除更广泛的概念(如人物或事件)。为此,我们提出概念遗忘(CU),作为新的模型遗忘需求。利用知识图谱表示模型内部知识,将CU定义为移除目标节点及其关联边。该图结构形式使遗忘过程更直观,并有助于设计更高效的方法。我们提出一种新方法:通过提示让模型生成目标概念的知识三元组及解释性语句,并对这些表示进行遗忘处理。该方法通过与模型内部知识表示对齐,实现更精确、全面的概念删除。在真实世界与合成数据集上的实验表明,本方法能有效实现概念级遗忘,同时保持无关知识不变。

原文摘要 · Abstract (English)

Machine Unlearning (MU) has recently attracted considerable attention as a solution to privacy and copyright issues in large language models (LLMs). Existing MU methods aim to remove specific target sentences from an LLM while minimizing damage to unrelated knowledge. However, these approaches require explicit target sentences and do not support removing broader concepts, such as persons or events. To address this limitation, we introduce Concept Unlearning (CU) as a new requirement for LLM unlearning. We leverage knowledge graphs to represent the LLM's internal knowledge and define CU as removing the forgetting target nodes and associated edges. This graph-based formulation enables a more intuitive unlearning and facilitates the design of more effective methods. We propose a novel method that prompts the LLM to generate knowledge triplets and explanatory sentences about the forgetting target and applies the unlearning process to these representations. Our approach enables more precise and comprehensive concept removal by aligning the unlearning process with the LLM's internal knowledge representations. Experiments on real-world and synthetic datasets demonstrate that our method effectively achieves concept-level unlearning while preserving unrelated knowledge.

大模型概念遗忘知识图谱隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。