为企事业单位定制RDF问答测试集，解决公开数据集不匹配私有数据的问题。
PIPE-RDF: An LLM-Assisted Pipeline for Enterprise RDF Benchmarking
- 用反向查询与模板生成构建符合企业模式的自然语言-查询对
- 生成450组问答对，修复后解析与执行成功率达100%
- 适合需要真实场景评估知识图谱模型的企业和研究者
企业依赖RDF知识图谱和SPARQL通过自然语言接口暴露运营数据，但现有公开KGQA基准无法反映专有模式、前缀或查询分布。我们提出PIPE-RDF，一个三阶段流程，利用反向查询、类别平衡的模板生成、检索增强提示、去重及基于执行的验证与修复，构建特定模式的NL-SPARQL基准。我们在一个固定模式的公司-位置子集（5,000家公司）上实例化该流程，生成覆盖九类任务的450个问题-SPARQL配对。修复后解析与执行有效率达100%，修复前各阶段有效性为96.5%-100%。报告实体多样性指标、模板覆盖率分析及成本分解以支持部署规划。我们发布结构化资源（CSV/JSONL、日志、图表）和运行指标，支持真实场景下的模型评估与系统规划。代码见https://github.com/suraj-ranganath/PIPE-RDF。
原文摘要 · Abstract (English)
Enterprises rely on RDF knowledge graphs and SPARQL to expose operational data through natural language interfaces, yet public KGQA benchmarks do not reflect proprietary schemas, prefixes, or query distributions. We present PIPE-RDF, a three-phase pipeline that constructs schema-specific NL-SPARQL benchmarks using reverse querying, category-balanced template generation, retrieval-augmented prompting, deduplication, and execution-based validation with repair. We instantiate PIPE-RDF on a fixed-schema company-location slice (5,000 companies) derived from public RDF data and generate a balanced benchmark of 450 question-SPARQL pairs across nine categories. The pipeline achieves 100% parse and execution validity after repair, with pre-repair validity rates of 96.5%-100% across phases. We report entity diversity metrics, template coverage analysis, and cost breakdowns to support deployment planning. We release structured artifacts (CSV/JSONL, logs, figures) and operational metrics to support model evaluation and system planning in real-world settings. Code is available at https://github.com/suraj-ranganath/PIPE-RDF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。