用语义分析优化多跳问答推理链，提升大模型知识更新效果
Knowledge Editing for Multi-Hop Question Answering Using Semantic Analysis
- 先语义分析再执行推理链，避免逻辑错误
- 在四个数据集上平均提升22.8%多跳问答准确率
- 适合需要精准推理的动态知识更新场景
大型语言模型（LLMs）需要轻量级方式更新过时信息。现有知识编辑方法在简单事实问答中表现良好，但在需要组合推理的多跳问答（MQA）任务中表现不佳。我们发现现有编辑器采用分解式策略，导致推理过程逻辑混乱。本文提出基于语义分析的知识编辑框架CHECK，受编译器与推理类比启发：如同源代码需先编译再执行，我们先对推理链进行语义分析，再执行回答。对存在语义错误的推理链，通过逻辑优化和高温重提示进行修正。在四个数据集上对比五种先进框架，CHECK实现平均22.8%的MQA准确率提升。
原文摘要 · Abstract (English)
Large Language Models (LLMs) require lightweight avenues of updating stored information that has fallen out of date. Knowledge Editing (KE) approaches have been successful in updating model knowledge for simple factual queries but struggle with handling tasks that require compositional reasoning such as multi-hop question answering (MQA). We observe that existing knowledge editors leverage decompositional techniques that result in illogical reasoning processes. In this paper, we propose a knowledge editor for MQA based on semantic analysis called CHECK. Our framework is based on insights from an analogy between compilers and reasoning using LLMs. Similar to how source code is first compiled before being executed, we propose to semantically analyze reasoning chains before executing the chains to answer questions. Reasoning chains with semantic errors are revised to ensure consistency through logic optimization and re-prompting the LLM model at a higher temperature. We evaluate the effectiveness of CHECK against five state-of-the-art frameworks on four datasets and achieve an average 22.8% improved MQA accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。