arXiv:2604.08284cs.CLcs.AI2026-04

针对大模型规则知识编辑难题,提出分层协同编辑方法。

Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models

  • 按符号、描述、实例分层设计编辑策略,突破单层干预局限。
  • 在四款主流模型上,规则理解提升50.19个百分点,实例迁移率提高13.91%。
  • 适用于需要精准推理的数学物理场景,适合模型可解释性研究者。

大型语言模型不仅存储孤立事实,还包含支持符号表达、自然语言解释与具体实例推理的规则知识。然而,现有编辑方法多针对事实级知识,假设单一局部干预即可完成修改,这一假设对规则级知识不成立——一条规则需在多种相互依赖的形式间保持一致。本文通过机制研究探索该问题,将RuleEdit基准从80条扩展至200条人工验证的数学与物理规则。细粒度因果追踪揭示:规则知识在变压器层中呈形式特异性分布——公式与描述集中于早期层,实例则更关联中间层。这表明规则知识并非均匀分布,因此无法通过单层或连续块干预可靠编辑。基于此,提出分布式多层编辑(DMLE):对公式和描述施加共享早期层更新,对实例采用独立中间层更新。在标准编辑指标上保持竞争力的同时,显著提升规则级编辑效果:平均而言,相比最强基线,在GPT-J-6B、Qwen2.5-7B、Qwen2-7B和LLaMA-3-8B上,实例可迁移性提升13.91个百分点,规则理解能力提升50.19个百分点。代码已公开于https://github.com/Pepper66/DMLE。

原文摘要 · Abstract (English)

Large language models store not only isolated facts but also rules that support reasoning across symbolic expressions, natural language explanations, and concrete instances. Yet most model editing methods are built for fact-level knowledge, assuming that a target edit can be achieved through a localized intervention. This assumption does not hold for rule-level knowledge, where a single rule must remain consistent across multiple interdependent forms. We investigate this problem through a mechanistic study of rule-level knowledge editing. To support this study, we extend the RuleEdit benchmark from 80 to 200 manually verified rules spanning mathematics and physics. Fine-grained causal tracing reveals a form-specific organization of rule knowledge in transformer layers: formulas and descriptions are concentrated in earlier layers, while instances are more associated with middle layers. These results suggest that rule knowledge is not uniformly localized, and therefore cannot be reliably edited by a single-layer or contiguous-block intervention. Based on this insight, we propose Distributed Multi-Layer Editing (DMLE), which applies a shared early-layer update to formulas and descriptions and a separate middle-layer update to instances. While remaining competitive on standard editing metrics, DMLE achieves substantially stronger rule-level editing performance. On average, it improves instance portability and rule understanding by 13.91 and 50.19 percentage points, respectively, over the strongest baseline across GPT-J-6B, Qwen2.5-7B, Qwen2-7B, and LLaMA-3-8B. The code is available at https://github.com/Pepper66/DMLE.

模型编辑规则推理多层结构可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。