显式词汇知识能显著提升问答系统性能,超越现有方法。
Lexicalization Is All You Need: Examining the Impact of Lexical Knowledge in a Compositional QALD System
- 构建可组合利用词汇知识的问答系统
- 相较最佳系统,微F1提升达35.8%
- 适合关注语义解析与知识融合的研究者
本文研究了词汇化在问答系统(QALD)中的作用。自然语言问题与SPARQL查询之间存在词汇鸿沟,即如何将问题中的词语映射到知识库的正确词汇元素。我们提出一种组合式问答系统,能够以组合方式显式利用词汇知识,推断问题的语义并生成对应SPARQL查询。实验表明,该系统在引入词汇知识后,微F1得分相比当前最佳系统在QALD-9上最高提升35.8%。相比之下,大模型对词汇知识利用能力有限,仅带来微弱改进,说明其缺乏基于成分意义的组合理解能力。结果表明,显式词汇化与组合性在问答系统中具有重要价值,为未来研究指明新方向。
原文摘要 · Abstract (English)
In this paper, we examine the impact of lexicalization on Question Answering over Linked Data (QALD). It is well known that one of the key challenges in interpreting natural language questions with respect to SPARQL lies in bridging the lexical gap, that is mapping the words in the query to the correct vocabulary elements. We argue in this paper that lexicalization, that is explicit knowledge about the potential interpretations of a word with respect to the given vocabulary, significantly eases the task and increases the performance of QA systems. Towards this goal, we present a compositional QA system that can leverage explicit lexical knowledge in a compositional manner to infer the meaning of a question in terms of a SPARQL query. We show that such a system, given lexical knowledge, has a performance well beyond current QA systems, achieving up to a $35.8\%$ increase in the micro $F_1$ score compared to the best QA system on QALD-9. This shows the importance and potential of including explicit lexical knowledge. In contrast, we show that LLMs have limited abilities to exploit lexical knowledge, with only marginal improvements compared to a version without lexical knowledge. This shows that LLMs have no ability to compositionally interpret a question on the basis of the meaning of its parts, a key feature of compositional approaches. Taken together, our work shows new avenues for QALD research, emphasizing the importance of lexicalization and compositionality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。