通过动态稀疏掩码实现精准神经元级知识编辑，减少错误累积。
Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
- 基于神经元功能归属识别两类关键知识神经元
- 熵引导的动态稀疏掩码仅修改少量参数即完成编辑
- 支持数千次连续编辑，保持高准确率与模型泛化能力
持续知识编辑使大语言模型可在不进行全量重训练的情况下持续、精确地更新过时知识。然而，现有方法在编辑过程中常积累误差，导致编辑准确率与泛化能力逐步下降。为此，我们提出神经元特异性掩码知识编辑（NMKE），一种细粒度的编辑框架，结合神经元层级归因与动态稀疏掩码机制。利用神经元功能归因，我们识别出两类关键知识神经元：跨提示一致激活的通用知识神经元，以及对特定提示响应的专用知识神经元。NMKE进一步引入熵引导的动态稀疏掩码，精准定位目标知识相关的神经元。该策略实现了仅需极少参数修改的神经元级知识编辑。上千次连续编辑实验表明，相比现有方法，NMKE在长期编辑中显著维持高编辑成功率并保护模型泛化能力。
原文摘要 · Abstract (English)
Lifelong knowledge editing enables continuous, precise updates to outdated knowledge in large language models (LLMs) without computationally expensive full retraining. However, existing methods often accumulate errors throughout the editing process, causing a gradual decline in both editing accuracy and generalization. To tackle this problem, we propose Neuron-Specific Masked Knowledge Editing (NMKE), a novel fine-grained editing framework that combines neuron-level attribution with dynamic sparse masking. Leveraging neuron functional attribution, we identify two key types of knowledge neurons, with knowledge-general neurons activating consistently across prompts and knowledge-specific neurons activating to specific prompts. NMKE further introduces an entropy-guided dynamic sparse mask, locating relevant neurons to the target knowledge. This strategy enables precise neuron-level knowledge editing with fewer parameter modifications. Experimental results from thousands of sequential edits demonstrate that NMKE outperforms existing methods in maintaining high editing success rates and preserving model general capabilities in lifelong editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。