让大模型更新知识后能更好推理，提升多跳任务准确率20%。
CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners
- 基于神经回路分析，动态引导模型构建新知识路径。
- 在MQuAKE数据集上多跳推理准确率提升20%。
- 仅需少量样本，内存开销低于现有方法，适合实际部署。
知识编辑（KE）可修改大语言模型中过时或错误的信息。现有方法虽能更新孤立事实，却难以将其推广至依赖更新知识的多跳推理任务。通过分析推理回路——大模型进行基于知识推理的神经通路，我们发现当前局限于单层或少数层的编辑方法（如MEMIT、WISE）未能有效整合更新知识到这些推理路径中。为此，我们提出CaKE（Circuit-aware Knowledge Editing），一种新方法，通过少量由回路分析指导的精选数据样本，激发模型为新知识构建合适的推理回路。实验表明，CaKE在相关推理任务中实现更准确、一致的知识使用，在MQuAKE数据集上多跳推理准确率平均提升20%，且内存消耗低于现有方法。代码与数据已开源：https://github.com/zjunlp/CaKE。
原文摘要 · Abstract (English)
Knowledge Editing (KE) enables the modification of outdated or incorrect information in large language models (LLMs). While existing KE methods can update isolated facts, they often fail to generalize these updates to multi-hop reasoning tasks that rely on the modified knowledge. Through an analysis of reasoning circuits -- the neural pathways LLMs use for knowledge-based inference, we find that current layer-localized KE approaches (e.g., MEMIT, WISE), which edit only single or a few model layers, inadequately integrate updated knowledge into these reasoning pathways. To address this limitation, we present CaKE (Circuit-aware Knowledge Editing), a novel method that enhances the effective integration of updated knowledge in LLMs. By only leveraging a few curated data samples guided by our circuit-based analysis, CaKE stimulates the model to develop appropriate reasoning circuits for newly incorporated knowledge. Experiments show that CaKE enables more accurate and consistent use of edited knowledge across related reasoning tasks, achieving an average improvement of 20% in multi-hop reasoning accuracy on the MQuAKE dataset while requiring less memory than existing KE methods. We release the code and data in https://github.com/zjunlp/CaKE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。