arXiv:2602.03696cs.LGcs.CL2026-02

提出CoRSA框架,解决大模型多轮知识编辑中的冲突与遗忘问题。

Conflict-Resolving and Sharpness-Aware Minimization for Generalized Knowledge Editing with Multiple Updates

  • 通过最小化损失曲率提升输入泛化性,增强多轮更新稳定性。
  • 在三个基准上平均比LoRA提升12.42%,多轮更新下遗忘率降低27.82%。
  • 适用于文本与代码场景,适合需持续更新知识的LLM应用。

大型语言模型依赖内部知识完成下游任务,保持其知识时效性至关重要。由于全量重训成本过高,已有研究探索了高效替代方案如模型编辑和参数高效微调。然而,这些方法常因输入泛化差、稳定性不足及知识冲突而失效。为此,我们提出CoRSA(冲突化解与锐度感知最小化)训练框架,一种支持多轮更新的参数高效、整体性知识编辑方法。CoRSA同时应对多项挑战:通过最小化损失曲率提升对不同输入形式的泛化能力并增强多轮更新稳定性;通过最大化新旧知识间的间隔来化解冲突。在三个广泛使用的事实编辑基准上,CoRSA在泛化性能上取得显著提升,平均绝对优于LoRA 12.42%,优于模型编辑方法10%。在多轮更新中,保持高更新效能的同时,相比LoRA将灾难性遗忘减少27.82%。该方法还扩展至代码领域,在Pass@5指标上优于最强基线5.48%。

原文摘要 · Abstract (English)

Large language models (LLMs) rely on internal knowledge to solve many downstream tasks, making it crucial to keep them up to date. Since full retraining is expensive, prior work has explored efficient alternatives such as model editing and parameter-efficient fine-tuning. However, these approaches often break down in practice due to poor generalization across inputs, limited stability, and knowledge conflict. To address these limitations, we propose the CoRSA (Conflict-Resolving and Sharpness-Aware Minimization) training framework, a parameter-efficient, holistic approach for knowledge editing with multiple updates. CoRSA tackles multiple challenges simultaneously: it improves generalization to different input forms and enhances stability across multiple updates by minimizing loss curvature, and resolves conflicts by maximizing the margin between new and prior knowledge. Across three widely used fact editing benchmarks, CoRSA achieves significant gains in generalization, outperforming baselines with average absolute improvements of 12.42% over LoRA and 10% over model editing methods. With multiple updates, it maintains high update efficacy while reducing catastrophic forgetting by 27.82% compared to LoRA. CoRSA also generalizes to the code domain, outperforming the strongest baseline by 5.48% Pass@5 in update efficacy.

知识编辑参数高效多轮更新大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。