让大模型动态选层编辑，精准更新知识不遗忘。
HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning
- 按需选择特定层进行编辑，避免全层扰动。
- 相比现有方法平均提升8.48%，仅扰动一半层。
- 适合长期部署中需持续更新知识的场景。
持续模型编辑（LME）旨在顺序修正已部署大语言模型中的过时或错误知识，同时最小化对无关输入的副作用。然而，现有方法通常对所有编辑实例使用静态且密集的模型层参数扰动，这违背直觉——我们假设不同知识分布在模型的不同层级。忽略这种层级特异性会降低新知识整合能力，并导致通用知识及先前编辑内容的灾难性遗忘。为此，我们提出HiEdit，一种分层强化学习框架，可自适应地为每次编辑任务识别最相关的知识层。通过实现动态、实例感知的层选择，并引入稀疏性内在奖励，HiEdit实现了精确、局部化的更新。在多种大模型上的实验表明，与竞争方法RLEdit相比，HiEdit平均性能提升8.48%，且每次编辑仅扰动一半层。代码已公开：https://github.com/yangfanww/hiedit。
原文摘要 · Abstract (English)
Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances. This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model. Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge. To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance. By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates. Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit. Our code is available at: https://github.com/yangfanww/hiedit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。