构建首个面向属性图的自然语言解析基准，支持复杂查询评估
ZOGRASCOPE: A New Benchmark for Semantic Parsing over Property Graphs
- 设计针对属性图的新型评测基准，支持Cypher查询
- 包含三类不同复杂度的标注查询，覆盖独立、组合与长度变化
- 适合知识图谱与大模型交互研究者使用
近年来，为知识图谱构建自然语言接口的需求日益迫切，因其能实现对其中信息的便捷高效访问。尤其属性图（PGs）作为表达复杂结构化信息的重要方式，在工业界应用愈发广泛。然而，现有语义解析研究对属性图的支持仍显不足，缺乏有效的评估资源。为此，本文提出ZOGRASCOPE，一个专为属性图设计的基准，支持用Cypher编写的查询。该基准包含多样化的手工标注查询，按复杂度分为三类：独立（iid）、组合（compositional）和长度（length）分区。同时，本文还开展了一系列实验，测试不同大模型在多种学习设置下的表现。
原文摘要 · Abstract (English)
In recent years, the need for natural language interfaces to knowledge graphs has become increasingly important since they enable easy and efficient access to the information contained in them. In particular, property graphs (PGs) have seen increased adoption as a means of representing complex structured information. Despite their growing popularity in industry, PGs remain relatively underrepresented in semantic parsing research with a lack of resources for evaluation. To address this gap, we introduce ZOGRASCOPE, a benchmark designed specifically for PGs and queries written in Cypher. Our benchmark includes a diverse set of manually annotated queries of varying complexity and is organized into three partitions: iid, compositional and length. We complement this paper with a set of experiments that test the performance of different LLMs in a variety of learning settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。