让大模型像人一样持续更新知识，且不遗忘旧内容。
Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs
- 在模型内部表示空间做干预，实现精准知识修改。
- 更新时仅动低维子空间，避免不同知识互相干扰。
- 适合需要长期维护知识的大模型应用者。
大语言模型在使用后常产生错误或过时内容，高效准确地更新知识而无需昂贵重训练是一大挑战，尤其在需持续处理复杂非结构化知识的场景中更为困难。本文提出RILKE（Representation Intervention for Lifelong KnowledgE Control），一种鲁棒且可扩展的方法，将知识控制视为模型表示空间中的干预操作。利用表示空间的表达能力，RILKE识别出两个关键特性：在训练中学习到对改写鲁棒、编辑局部化的模块，使每次更新局限于低维子空间以最小化跨编辑干扰；在推理时通过查询自适应路由选择合适模块引导生成。在LLaMA和Qwen模型上，RILKE在大规模基准测试中表现出高编辑成功率与强改写泛化能力，同时保持通用性能，仅需少量内存开销。结果表明，RILKE是大模型持续知识控制的有效且可扩展方案。
原文摘要 · Abstract (English)
Large language models (LLMs) often produce incorrect or outdated content after being employed. Efficient and accurate knowledge updates without costly retraining are a major challenge. This problem is particularly challenging in lifelong settings, where complex, unstructured knowledge must coexist without interference. We introduce RILKE (Representation Intervention for Lifelong KnowledgE Control), a robust and scalable method that treats knowledge control as interventions within the model's representation space. Leveraging representation-space expressiveness, we identify two key properties enabling RILKE to achieve fine-grained control over complex, unstructured knowledge while maintaining general utility with frozen base weights. During training, RILKE learns paraphrase-robust and edit-localized modules that limit each update to a low-dimensional subspace to minimize cross-edit interference. At inference, a query-adaptive router selects the appropriate module to guide the model's generation. Across LLaMA and Qwen models, RILKE scales effectively to large-scale benchmarks, demonstrating high edit success and strong paraphrase generalization while preserving general utility with modest memory overhead. These results show RILKE is an effective and scalable solution for lifelong knowledge control in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。