现有知识图谱复杂查询数据集其实很简单，多数可简化为链接预测。
Is Complex Query Answering Really Complex?
- 发现98%的查询可转化为单跳链接预测问题
- 顶尖模型在无法简化的多跳查询上性能大幅下降
- 提出新基准测试，更贴近真实知识图谱推理场景
知识图谱上的复杂查询回答（CQA）被视为一项具有挑战性的推理任务。本文指出，当前的CQA评测基准可能并不如想象中复杂，因为其构建方式扭曲了该领域进展的真实情况。例如，我们发现这些基准中大多数查询（某些查询类型高达98%）可被简化为更简单的任务，如仅需预测一条边的链接预测。当将当前最先进的CQA模型应用于无法简化的查询时，其性能显著下降。因此，我们提出一组更具挑战性的新基准，包含需要模型进行多跳推理且更贴近真实知识图谱构建方式的查询。通过系统性实证研究，新基准表明当前方法仍远未达到理想水平。
原文摘要 · Abstract (English)
Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be as complex as we think, as the way they are built distorts our perception of progress in this field. For example, we find that in these benchmarks, most queries (up to 98% for some query types) can be reduced to simpler problems, e.g., link prediction, where only one link needs to be predicted. The performance of state-of-the-art CQA models decreases significantly when such models are evaluated on queries that cannot be reduced to easier types. Thus, we propose a set of more challenging benchmarks composed of queries that require models to reason over multiple hops and better reflect the construction of real-world KGs. In a systematic empirical investigation, the new benchmarks show that current methods leave much to be desired from current CQA methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。