首个覆盖主流图查询语言的文本转图查询基准，助力大模型更好理解自然语言图查询。
Text2GraphQuery-Bench: A Text to Graph Query Benchmark
- 构建跨34库13领域的26万+问答对，支持多语言图查询生成
- 发现小样本提示可显著弥补模型对新语言的不熟悉，提升语法正确率
- 适用于图数据库研究者与大模型应用开发者，推动自然语言交互优化
图模型在复杂关系数据中至关重要。与统一且普及的SQL不同，图查询语言（如Cypher、GQL、SQL/PGQ）种类多样且用户不熟悉，学习使用门槛高。文本转图查询系统通过将自然语言转化为可执行图查询，使大模型成为图数据库管理系统的接口。现有基准在语言覆盖、合成方式和评估全面性上存在局限。本文提出Text2GraphQuery-Bench，首个涵盖主流声明式属性图查询语言（Cypher、GQL、SQL/PGQ）的基准，包含267,276个（问题，图查询）对，覆盖34个数据库和13个领域。其构建支持异构资源适配与领域感知合成，基于Graph-IR的设计可快速扩展至新语言。评估协议在图原生难度、问题抽象化和模式别名条件下报告语法正确率（Grammar）、GLEU、相似度（Similarity）和执行准确率（EX）。8个大模型实验表明：(i) 存在显著语言差距——零样本下GQL和SQL/PGQ语法远低于Cypher，但少样本提示可大幅恢复；(ii) 微调8B模型达到甚至超过零样本大模型表现，说明主要瓶颈是语言不熟悉而非模型能力；(iii) 随着监督增加，语法错误减少，瓶颈转向GQL的聚合逻辑与SQL/PGQ的模式关联；(iv) 问题抽象度越高，执行准确率下降，因意图到模式映射困难，而模式别名影响较小；(v) 从Easy到Extra Hard，执行准确率持续下降，其中Extra Hard仍是核心挑战。
原文摘要 · Abstract (English)
Graph models are fundamental to data analysis in domains rich with complex relationships. Unlike SQL, which benefits from a rel- atively unified standard and widespread familiarity, graph query languages are diverse (e.g., Cypher, GQL, SQL/PGQ) and far less fa- miliar to most users, making them significantly harder to learn and use. Text-to-Graph-Query systems address this barrier by trans- lating natural language into executable graph queries, enabling LLMs to serve as interfaces for Graph Database Management Systems (GDBMS). Existing benchmarks are limited in language coverage, rely on rigid synthesis, and lack comprehensive evaluation. We present Text2GraphQuery-Bench, the first benchmark covering all mainstream declarative property graph query languages (Cypher, GQL, and SQL/PGQ). It contains 267,276 (Question, Graph Query) pairs across 34 databases and 13 domains. Its construction supports adaptation from heterogeneous resources and domain-aware synthesis, while its Graph-IR-based design enables rapid extension to new languages. The evaluation protocol reports Grammar, GLEU, Similarity, and EX under graph-native difficulty, question abstraction, and schema aliasing. Experiments on 8 LLMs reveal: (i) a significant language gap exists - zero-shot GQL and SQL/PGQ Grammar is far below Cypher, yet few-shot prompting largely recovers it; (ii) fine-tuning an 8B model reaches or exceeds zero-shot large models, indicating unfamiliarity - rather than model capacity - is the primary barrier; (iii) as supervision increases, syntax errors recede, shifting bottlenecks to aggregation logic in GQL and schema linking in SQL/PGQ; (iv) higher question abstraction degrades EX due to intent-to-schema grounding issues, while schema aliasing has minimal impact; (v) EX consistently degrades from Easy to Extra Hard, with Extra Hard remaining a persistent bottleneck. *(Due to arXiv constraints, this abstract is shortened. See PDF for the full version.)*
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。