通过分析模型内部计算机制，实现无需修改权重的知识精准更新。
The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
- 基于编辑后激活值对比，定位成功编辑的关键计算区域。
- 在中后期注意力层增强新知识，同时抑制旧知识的激活。
- 无需改模型参数，适配多种大模型且效果稳定。
大语言模型被广泛用作知识库，但保持其知识更新需针对性的知识编辑（KE）。然而，编辑在模型内部如何实现仍不明确。本文采用神经元级知识归因（NLKA）的机制视角研究KE。不同于以往关注编辑前因果追踪的工作，我们采用编辑后归因——对比成功与失败编辑，识别出编辑成功时发生变化的计算模块。在多种代表性KE方法中，我们发现一致模式：中后期注意力主要促进新目标，而注意力与前馈网络协同抑制原事实。基于此，我们提出MEGA方法，通过在归因对齐区域进行注意力残差干预，无需修改模型权重即可实现知识编辑。在CounterFact和Popular数据集上，MEGA在GPT2-XL和LLaMA2-7B上均展现出优异的编辑性能。结果表明，编辑后归因可从分析工具升级为工程信号，精准定位编辑生效位置与机制，从而驱动MEGA实现可靠、架构无关的知识编辑。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE). However, it remains unclear how edits are implemented inside the model once applied. In this work, we take a mechanistic view of KE using neuron-level knowledge attribution (NLKA). Unlike prior work that focuses on pre-edit causal tracing and localization, we use post-edit attribution -- contrasting successful and failed edits -- to isolate the computations that shift when an edit succeeds. Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact. Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions without modifying model weights. On CounterFact and Popular, MEGA achieves strong editing performance across KE metrics on GPT2-XL and LLaMA2-7B. Overall, our results elevate post-edit attribution from analysis to engineering signal: by pinpointing where and how edits take hold, it powers MEGA to deliver reliable, architecture-agnostic knowledge edits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。