动态选层+梯度投影，实现无需历史数据的高效模型持续更新。
LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing
- 根据独立性准则动态选择更新层，提升灵活性
- 梯度投影到权重零空间，最高提升14%准确率
- 无需历史数据和预处理，适合长期知识迭代
持续知识编辑旨在高效、顺序地更新语言模型，以应对新知识或模型错误，同时保持对旧知识的良好性能。现有方法存在两个问题：固定层更新缺乏灵活性，且需访问历史知识与复杂预处理获取数据统计。为此，我们提出LOKI，通过希尔伯特-施密特独立性准则动态选择更新层，并将梯度更新投影至模型权重的零空间，从而避免对历史知识的依赖。实验表明，LOKI在多种场景下均优于现有方法，平均准确率最高提升14%。
原文摘要 · Abstract (English)
Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge. One unresolved challenge is that existing methods modify a fixed set of layers for all new knowledge samples, reducing flexibility and increasing catastrophic forgetting. Another is requiring access to previous knowledge and extensive pre-processing to obtain data statistics. To address these challenges, we introduce LOKI, a novel approach that uses dynamic layer selection based on the Hilbert-Schmidt Independence Criterion and projects gradient updates onto the null-space of the model weights, bypassing the requirement for previous knowledge access. We show that LOKI achieves superior performance to existing approaches across a wide variety of experiments, achieving up to a 14\% improvement in average accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。