让大模型分步挖外部知识，解决复杂隐含问题
Gradually Excavating External Knowledge for Implicit Complex Question Answering
- 大模型分步主动查询外部知识，逐步推理
- 在StrategyQA上达78.17%准确率，参数量不足对手6%
- 适合需要深度知识推理的开放域问答任务
大语言模型虽具类人能力，但在开放域隐含问题问答中仍受限于未覆盖或过时的知识，以及单次生成导致的全面性不足。为此，本文提出一种渐进式外部知识挖掘框架，使大模型在解题过程中迭代主动获取外部信息，并基于历史知识进行推理。每一步中，模型选择查询外部知识或执行逻辑推理等动作，逐步逼近答案。该方法可有效利用即插即用的外部知识，并动态调整解题策略。在StrategyQA数据集上的实验表明，该方法以不到6%的参数量达到78.17%的准确率，刷新约10B规模大模型的新纪录。
原文摘要 · Abstract (English)
Recently, large language models (LLMs) have gained much attention for the emergence of human-comparable capabilities and huge potential. However, for open-domain implicit question-answering problems, LLMs may not be the ultimate solution due to the reasons of: 1) uncovered or out-of-date domain knowledge, 2) one-shot generation and hence restricted comprehensiveness. To this end, this work proposes a gradual knowledge excavation framework for open-domain complex question answering, where LLMs iteratively and actively acquire external information, and then reason based on acquired historical knowledge. Specifically, during each step of the solving process, the model selects an action to execute, such as querying external knowledge or performing a single logical reasoning step, to gradually progress toward a final answer. Our method can effectively leverage plug-and-play external knowledge and dynamically adjust the strategy for solving complex questions. Evaluated on the StrategyQA dataset, our method achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA for ~10B-scale LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。