用大模型融合结构化与文本数据，提升学术问答准确率
Contri(e)ve: Context + Retrieve for Scholarly Question Answering
- 分两步提取学术知识图谱与维基文本中的上下文信息
- 在Scholarly-QALD数据集上达到40% F1分数
- 适合需要融合多源学术数据的智能问答研究者
学术传播领域知识丰富但格式非结构化，传统文档检索难以有效提取信息。学术知识图谱通过语义网络表示文献，提供隐含洞察、摘要和查询便利性。尽管如此，部分知识仍以非结构化文本形式存在，需混合解决方案。本文提出基于开源大模型Llama3.1的两阶段方法：首先从DBLP、SemOpenAlex知识图谱及维基百科文本中提取与问题相关的上下文；其次通过提示工程优化LLM的信息检索性能。实验在Scholarly-QALD数据集上取得40% F1分数，并分析了模型出现的异常回答现象。
原文摘要 · Abstract (English)
Scholarly communication is a rapid growing field containing a wealth of knowledge. However, due to its unstructured and document format, it is challenging to extract useful information from them through conventional document retrieval methods. Scholarly knowledge graphs solve this problem, by representing the documents in a semantic network, providing, hidden insights, summaries and ease of accessibility through queries. Naturally, question answering for scholarly graphs expands the accessibility to a wider audience. But some of the knowledge in this domain is still presented as unstructured text, thus requiring a hybrid solution for question answering systems. In this paper, we present a two step solution using open source Large Language Model(LLM): Llama3.1 for Scholarly-QALD dataset. Firstly, we extract the context pertaining to the question from different structured and unstructured data sources: DBLP, SemOpenAlex knowledge graphs and Wikipedia text. Secondly, we implement prompt engineering to improve the information retrieval performance of the LLM. Our approach achieved an F1 score of 40% and also observed some anomalous responses from the LLM, that are discussed in the final part of the paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。