通过神经元集合精准编辑大模型知识，避免参数耦合问题。
Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble
- 用神经元集合表征特定知识，减少参数修改频率。
- 通过梯度归因定位关键参数，提升编辑精度。
- 支持跨层动态交互，适合需要高精度知识更新的场景。
现实世界知识持续演进，确保大语言模型知识的时效性与准确性至关重要。现有知识编辑方法面临参数定位耦合、定位不精确及层间动态交互缺失等问题。本文提出一种新型知识编辑方法——知识神经元集成（KNE）。KNE将一组编码特定知识的神经元视为知识神经元集合，缓解参数定位耦合带来的频繁参数修改问题。通过计算各层参数的梯度归因分数，提升参数定位的精度与准确性。编辑过程中仅对知识神经元集合相关的梯度与损失进行计算，并相应执行误差反向传播，确保参数间的动态交互与协同更新。在三个常用知识编辑数据集上的实验结果表明，KNE显著提升了知识编辑的准确性，在可移植性与局部性指标上达到或超过最佳基线方法的表现。
原文摘要 · Abstract (English)
As real-world knowledge is constantly evolving, ensuring the timeliness and accuracy of a model's knowledge is crucial. This has made knowledge editing in large language models increasingly important. However, existing knowledge editing methods face several challenges, including parameter localization coupling, imprecise localization, and a lack of dynamic interaction across layers. In this paper, we propose a novel knowledge editing method called Knowledge Neuronal Ensemble (KNE). A knowledge neuronal ensemble represents a group of neurons encoding specific knowledge, thus mitigating the issue of frequent parameter modification caused by coupling in parameter localization. The KNE method enhances the precision and accuracy of parameter localization by computing gradient attribution scores for each parameter at each layer. During the editing process, only the gradients and losses associated with the knowledge neuronal ensemble are computed, with error backpropagation performed accordingly, ensuring dynamic interaction and collaborative updates among parameters. Experimental results on three widely used knowledge editing datasets show that the KNE method significantly improves the accuracy of knowledge editing and achieves, or even exceeds, the performance of the best baseline methods in portability and locality metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。