arXiv:2510.13851cs.CLcs.LG2025-10ACL被引 5

解决大模型知识编辑时的干扰问题,实现高效稳定更新。

EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing

  • 通过逐次零空间对齐,保留旧知识和新修改内容。
  • 在多轮编辑中保持输出一致性,最长可支持多轮无退化。
  • 适合需要频繁更新知识的大模型应用场景。

大型语言模型(LLMs)需要持续更新以修正过时或错误的知识。模型编辑作为一种无需全量重训练即可实现精准修改的范式应运而生。现有方法多基于‘定位-编辑’框架,但在连续编辑场景下存在显著局限,易引发灾难性干扰——即新编辑会破坏先前已集成的更新并损害保留知识。为此,本文提出EvoEdit,一种通过逐次零空间对齐缓解灾难性干扰的新编辑策略,实现稳定且高效的模型编辑。通过为每项新编辑执行序列化的零空间对齐,EvoEdit同时保留原始与先前修改的知识表征,并在长序列编辑中维持对保留知识的输出不变性,有效抑制干扰。在真实世界连续知识编辑基准上的评估显示,EvoEdit性能优于或相当于当前最先进方法,最快可达3.53倍加速。结果表明,在动态演化的信息环境中,需发展更严谨的模型设计范式,而EvoEdit提供了兼具理论保障与实用性的简单有效方案。

原文摘要 · Abstract (English)

Large language models (LLMs) require continual updates to rectify outdated or erroneous knowledge. Model editing has emerged as a compelling paradigm for introducing targeted modifications without the computational burden of full retraining. Existing approaches are mainly based on a locate-then-edit framework. However, in sequential editing contexts, where multiple updates are applied over time, they exhibit significant limitations and suffer from catastrophic interference, i.e., new edits compromise previously integrated updates and degrade preserved knowledge. To address these challenges, we introduce EvoEdit, a novel editing strategy that mitigates catastrophic interference through sequential null-space alignment, enabling stable and efficient model editing. By performing sequential null-space alignment for each incoming edit, EvoEdit preserves both original and previously modified knowledge representations and maintains output invariance on preserved knowledge even across long edit sequences, effectively mitigating interference. Evaluations on real-world sequential knowledge-editing benchmarks show that EvoEdit achieves better or comparable performance than prior state-of-the-art locate-then-edit techniques, with up to 3.53 times speedup. Overall, these results underscore the necessity of developing more principled approaches for designing LLMs in dynamically evolving information settings, while providing a simple yet effective solution with strong theoretical guarantees.

知识编辑大模型零空间对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。