提出可长期保持知识的序列编辑框架，解决大模型多次修正后性能下降问题。
LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editing
- 用李雅普诺夫优化与排队论建模序列编辑，分解为可解子问题
- 支持超万次编辑，平均效果比现有方法高11.89%，且通用能力不退化
- 首个有理论保证的编辑框架，适合需要持续更新知识的场景
大语言模型常包含错误或过时的知识，需通过模型编辑实现精准更新。然而当前主流的定位-编辑方法在连续编辑过程中存在性能逐步下降的问题，源于长期知识保留机制不足。为此，本文将序列编辑建模为约束随机规划问题，针对累积保留误差约束和渐进式揭示的编辑任务，提出 extbf{LyapLock} 框架。该框架融合排队论与李雅普诺夫优化，将长期约束规划分解为可计算的分步子问题，实现高效求解。这是首个具有严格理论保障的模型编辑框架，在保证长期知识保留的前提下，实现渐近最优的编辑性能。实验表明，该框架可支持超过10,000次连续编辑，稳定通用能力，并使平均编辑效率相比最优基线提升11.89%。此外，其可增强现有基线方法性能。代码已开源：https://github.com/caskcsg/LyapLock。
原文摘要 · Abstract (English)
Large Language Models often contain factually incorrect or outdated knowledge, giving rise to model editing methods for precise knowledge updates. However, current mainstream locate-then-edit approaches exhibit a progressive performance decline during sequential editing, due to inadequate mechanisms for long-term knowledge preservation. To tackle this, we model the sequential editing as a constrained stochastic programming. Given the challenges posed by the cumulative preservation error constraint and the gradually revealed editing tasks, \textbf{LyapLock} is proposed. It integrates queuing theory and Lyapunov optimization to decompose the long-term constrained programming into tractable stepwise subproblems for efficient solving. This is the first model editing framework with rigorous theoretical guarantees, achieving asymptotic optimal editing performance while meeting the constraints of long-term knowledge preservation. Experimental results show that our framework scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89\% over SOTA baselines. Furthermore, it can be leveraged to enhance the performance of baseline methods. Our code is released on https://github.com/caskcsg/LyapLock.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。