解决大模型连续编辑时知识泄露和性能下降问题
BetaEdit: Null-Space Constrained Sequential Model Editing
- 在零空间约束基础上引入历史感知更新，控制知识泄露
- 在三大模型上实现大规模连续编辑任务性能领先
- 适合需要长期精准修改大模型知识的科研与工程场景
基于零空间的方法通过将参数更新限制在预训练知识表示的零空间中，以保持模型原有行为，但实际应用中依赖近似零空间，导致知识泄露，并在连续编辑中出现严重性能退化。尽管近期工作表明历史感知策略能缓解性能下降，其内在原因仍不明确。本文首先揭示现有零空间方法中的知识泄露问题，进而分析历史感知更新为何能在长序列编辑中同时保持编辑效果与通用能力。基于此，我们提出BetaEdit，一种能有效控制知识泄露并融合历史感知更新的改进框架。在三个大型语言模型、两个标准基准上的大量实验表明，BetaEdit在大规模连续编辑任务中持续优于先前方法。代码已开源：https://github.com/lbq8942/BetaEdit。
原文摘要 · Abstract (English)
Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。