通过聚焦关键特征子空间，实现精准知识编辑且不扰动模型其他部分。
SUIT: Knowledge Editing with Subspace-Aware Key-Value Mappings
- 只在与编辑相关的特征子空间内更新键值映射。
- 在多个大模型上实现高编辑效果同时显著提升知识保留率。
- 适合需要低干扰、高可靠性的知识修正场景。
知识编辑旨在高效纠正语言模型中的事实错误。现有定位-编辑方法通过调整MLP层权重来改变输入向量（键）与输出向量（值）的映射关系以实现知识更新。然而，若未对键值向量施加严格约束，此类更新常导致超出目标编辑范围的模型扰动，这是多数已有方法的共性问题。为此，我们提出子空间感知的知识编辑方法SUIT，仅在与编辑任务相关的关键特征子空间内计算键值向量。在LLaMA3、GPT-J和Qwen2.5模型上的实证结果表明，SUIT在保持高编辑性能的同时，显著优于强基线方法，大幅提升了知识保留能力。分析显示，SUIT有效减少了隐藏状态的非预期扰动，并将更新集中在更利于编辑的方向上。这些发现确立了识别编辑关键子空间作为实现可靠、低扰动知识编辑的核心原则。代码已开源：https://github.com/holi-lab/SUIT。
原文摘要 · Abstract (English)
Knowledge editing aims to efficiently correct factual errors in language models. Widely used locate-then-edit methods update an MLP layer by adjusting its weights to change the mapping between the layer's input vector (key) and output vector (value), thereby editing the model's knowledge. As this update is driven by key and value vectors, obtaining these vectors without careful constraints causes significant model perturbations beyond the targeted edit, a common issue in many prior knowledge editing methods. To address this, we propose Subspace Knowledge Edit (SUIT), which computes key and value vectors only within the subspace of critical features relevant to the edit. Our empirical results on LLaMA3, GPT-J, and Qwen2.5 models show that SUIT dramatically improves knowledge preservation over strong baselines while maintaining high editing performance. These results support the claim that SUIT successfully identifies the critical subspace for the edit. Beyond quantitative gains, our analyses show that SUIT reduces unintended perturbations in hidden states while confining updates to directions that are more effective for editing. Taken together, these findings establish edit-critical subspace identification as a key principle for reliable, low-perturbation knowledge editing. Our code is available at https://github.com/holi-lab/SUIT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。