用分步拆解法提升大模型答复杂题能力,效果显著且稳定。
Research on Multi-hop Inference Optimization of LLM Based on MQUAKE Framework
- 将复杂问题拆成多步推理链,结合知识图谱增强理解。
- 未微调时,拆解法准确率远超直接回答,差距达15%以上。
- 微调后仍保持优势,适合需要精准推理的场景。
准确回答复杂问题一直是大语言模型(LLMs)的挑战。本文基于MQUAKE框架,提出一种多跳问题分解方法,利用LLAMA3模型系统研究该方法在知识图谱中对模型理解与推理准确性的影响,涵盖训练前与训练后两种情况。实验中,我们将MQUAKE-T数据集分别转换为单跳(直接回答复杂问题)和多跳(通过多跳分解构建)两种格式,并对LLAMA3模型进行微调(采用LoRA方法)后进行推理测试。结果显示,在未微调情况下,基于多跳分解的方法显著优于直接回答,性能提升超过15%;微调后两种方法均优于原始基线,但多跳分解方法始终保持优势。这验证了该方法在训练前后均有效,能切实增强大模型处理复杂问题的能力。
原文摘要 · Abstract (English)
Accurately answering complex questions has consistently been a significant challenge for Large Language Models (LLMs). To address this, this paper proposes a multi-hop question decomposition method for complex questions, building upon research within the MQUAKE framework. Utilizing the LLAMA3 model, we systematically investigate the impact of multi-hop question decomposition within knowledge graphs on model comprehension and reasoning accuracy, both before and after model training. In our experiments, we systematically partitioned and converted the MQUAKE-T dataset into two distinct formats: a single-hop dataset designed for directly answering complex questions, and a multi-hop dataset constructed using the multi-hop question decomposition method. We then fine-tuned the LLAMA3 model on these datasets and conducted inference tests. Our results demonstrate that, without fine-tuning the LLM, the prediction performance based on the multi-hop question decomposition method significantly outperforms the method of directly answering complex questions. After fine-tuning using the LoRA (Low-Rank Adaptation) method, the performance of both approaches improved compared to the untrained baseline. Crucially, the method utilizing multi-hop decomposition consistently maintained its superiority. These findings validate the effectiveness of the multi-hop decomposition method both before and after training, demonstrating its capability to effectively enhance the LLM's ability to answer complex questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。