通过稀疏电路锚点神经元实现持续知识编辑，避免模型遗忘与崩溃
SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing
- 基于稀疏编码器构建知识电路，实现机制可解释的精准编辑
- 在3000次连续编辑后仍保持MMLU和GSM8K等基准性能稳定
- 适合需要长期更新知识且不希望模型退化的应用场景
大型语言模型在顺序知识编辑过程中常出现灾难性遗忘与模型崩溃。这一问题源于主流的密集编辑范式，该范式将模型视为黑箱，依赖粗粒度参数干预，不可避免地破坏已有知识。为此，我们提出SCAN（基于稀疏电路锚点神经元的稀疏编辑框架），通过稀疏编码器构建知识电路，将编辑转化为机制感知的操作。在Gemma2、Qwen3和Llama3.1上，针对CounterFact、ZsRE和WikiFactDiff数据集的实验表明，SCAN在完成3000次连续编辑后，仍能保持MMLU和GSM8K等基准测试的模型完整性，而其他方法随编辑次数增加逐渐恶化，最终导致模型崩溃。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。