arXiv:2508.04012cs.CLcs.AI2025-08中稿 · WWW2026被引 2

高效多步元学习模型编辑方法,解决小样本下知识更新难、训练慢问题。

EMSEdit: Efficient Multi-Step Meta-Learning-based Model Editing

  • 采用多步反向传播捕捉梯度激活模式,提升小样本编辑效果
  • 多步编辑策略使性能在低数据条件下显著优于现有方法
  • 引入范数正则化,兼顾知识保留与训练效率,适合实际部署

大型语言模型(LLMs)驱动众多AI应用,但知识更新成本高昂。模型编辑通过针对性参数修改提供轻量级替代方案,基于元学习的模型编辑(MLME)展现出强有效性和高效性。然而,我们发现MLME在低数据场景下表现不佳,且因使用KL散度导致训练成本高。为此,提出高效多步模型编辑方法EMSEdit,利用多步反向传播(MSBP)捕捉编辑样本中的梯度-激活映射模式,对每样本执行多步编辑以增强有限数据下的编辑性能,并引入基于范数的正则化,在保持未编辑知识的同时提升训练效率。在两个数据集和三个LLM上的实验表明,EMSEdit在序列与批量编辑中均持续优于当前最佳方法。此外,MSBP可无缝集成至现有方法中进一步提升性能。多跳推理编辑任务实验验证了其处理复杂编辑的鲁棒性,消融实验确认各设计组件的有效性。代码已开源。

原文摘要 · Abstract (English)

Large Language Models (LLMs) power numerous AI applications, yet updating their knowledge remains costly. Model editing provides a lightweight alternative through targeted parameter modifications, with meta-learning-based model editing (MLME) demonstrating strong effectiveness and efficiency. However, we find that MLME struggles in low-data regimes and incurs high training costs due to the use of KL divergence. To address these issues, we propose $\textbf{E}$fficient $\textbf{M}$ulti-$\textbf{S}$tep $\textbf{Edit (EMSEdit)}$, which leverages multi-step backpropagation (MSBP) to effectively capture gradient-activation mapping patterns within editing samples, performs multi-step edits per sample to enhance editing performance under limited data, and introduces norm-based regularization to preserve unedited knowledge while improving training efficiency. Experiments on two datasets and three LLMs show that EMSEdit consistently outperforms state-of-the-art methods in both sequential and batch editing. Moreover, MSBP can be seamlessly integrated into existing approaches to yield additional performance gains. Further experiments on a multi-hop reasoning editing task demonstrate EMSEdit's robustness in handling complex edits, while ablation studies validate the contribution of each design component. Our code is available at https://github.com/xpq-tech/emsedit.

模型编辑元学习大模型高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。