arXiv:2512.04753cs.CL2025-12被引 1

提出编辑后加固框架,让大模型改错更准且不丢原有能力。

EtCon: Edit-then-Consolidate for Reliable Knowledge Editing

  • 先精准修改知识层,再通过强化学习对齐生成行为
  • 在多个数据集上编辑准确率超基线15%以上,且原能力下降<5%
  • 适合需要可靠知识更新的工业级大模型应用

知识编辑旨在不进行全量重训练的情况下更新大语言模型中的特定事实。以往方法试图调整模型的知识层,在受控的教师强制评估中表现良好,但在真实场景的自回归生成中仍存在局限,严重制约实际应用。我们实证分析发现两个问题:(1) 多数方法在注入新知识后会削弱预训练能力;(2) 参数化知识与推理时的自回归生成行为之间存在偏差。为此,我们提出EtCon——一种编辑后加固范式,将目标编辑与后编辑巩固相结合。框架包含两阶段:(1) 靶向近端监督微调(TPSFT)在控制策略漂移的前提下完成精确编辑;(2) 组相对策略优化(GRPO)通过对齐自回归轨迹来巩固编辑效果。大量实验表明,EtCon显著提升了编辑可靠性与现实泛化能力,同时更好保留了预训练能力。

原文摘要 · Abstract (English)

Knowledge editing aims to update specific facts in large language models (LLMs) without full retraining. Prior efforts sought to tune the knowledge layers of LLMs, achieving improved performance in controlled, teacher-forced evaluations. However, they still encounter challenges in real-world autoregressive generation scenarios, which greatly limit their practical applicability. Our empirical analysis reveals two issues: (1) Most methods degrade pre-trained capabilities after injecting new knowledge; (2) They may exhibit a discrepancy between stored parametric knowledge and inference-time autoregressive generation behavior. To this end, we propose EtCon, an edit-then-consolidate paradigm that couples targeted edits with post-edit consolidation. Specifically, our framework comprises two stages: (1) Targeted Proximal Supervised Fine-Tuning (TPSFT) performs a constrained targeted edit to update parametric knowledge while controlling policy drift. (2) Group Relative Policy Optimization (GRPO) consolidates the edit by aligning autoregressive trajectories with the intended fact. Extensive experiments demonstrate that our EtCon improves editing reliability and real-world generalization, while better preserving pre-trained capabilities.

知识编辑大模型强化学习可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。