arXiv:2509.20977cs.LGcs.CL2025-09被引 5

通过电路发现技术精准定位应删除或保留的神经元,避免模型遗忘过度或记忆残留。

CLUE: Conflict-guided Localization for LLM Unlearning Framework

  • 利用电路发现识别遗忘与保留的神经元电路
  • 将神经元电路转为逻辑表达式,精确判断每个神经元去留
  • 适合需要安全擦除数据又不破坏模型能力的研究者

大语言模型的遗忘机制旨在消除有害数据的影响,同时保留无关信息。传统方法依赖遗忘集和保留集,但现有基于定位的方法无法区分负责遗忘有害知识与保留核心能力的神经元,常将其视为一个整体,导致统一干预,引发灾难性遗忘或目标知识清除不彻底。为此,本文提出冲突引导定位框架(CLUE),采用机制可解释性中的电路发现技术,识别出遗忘与保留电路,将这些电路转化为合取范式(CNF),通过求解布尔可满足性,确定每个神经元是否应被遗忘或保留,并据此实施针对性微调。大量实验表明,相比现有定位方法,CLUE在遗忘效果和保留能力方面均表现更优,实现更精确的神经元级控制。

原文摘要 · Abstract (English)

The LLM unlearning aims to eliminate the influence of undesirable data without affecting causally unrelated information. This process typically involves using a forget set to remove target information, alongside a retain set to maintain non-target capabilities. While recent localization-based methods demonstrate promise in identifying important neurons to be unlearned, they fail to disentangle neurons responsible for forgetting undesirable knowledge or retaining essential skills, often treating them as a single entangled group. As a result, these methods apply uniform interventions, risking catastrophic over-forgetting or incomplete erasure of the target knowledge. To address this, we turn to circuit discovery, a mechanistic interpretability technique, and propose the Conflict-guided Localization for LLM Unlearning framEwork (CLUE). This framework identifies the forget and retain circuit composed of important neurons, and then the circuits are transformed into conjunctive normal forms (CNF). The assignment of each neuron in the CNF satisfiability solution reveals whether it should be forgotten or retained. We then provide targeted fine-tuning strategies for different categories of neurons. Extensive experiments demonstrate that, compared to existing localization methods, CLUE achieves superior forget efficacy and retain utility through precise neural localization.

大模型遗忘神经元定位可解释性逻辑推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。