利用执行错误反馈提升Cypher查询生成准确率
RAS: Reflection-Augmented Scaling with In-Context Learning for Executable Cypher Query Generation
- 通过上下文学习整合执行错误反馈，动态优化生成过程
- 在5次尝试下将查询执行错误率降低41%~50%
- 适合需要高可执行性查询的图数据库应用开发者
推理时的规模扩展可降低结构化查询生成中的错误，但如何分配计算资源仍缺乏研究。本文研究Text2Cypher任务，即语言模型生成可执行的Cypher查询以作用于属性图数据库。非可执行查询属于独立于语义不准确的语法错误：语法错误会触发数据库自动生成的错误信息。这些错误信息通常在推理时被忽略，未被用于上下文学习（ICL）。我们对比两种推理方法：无记忆重采样的独立扩展（IS）与基于执行反馈进行上下文学习的反射增强扩展（RAS）。在三个Neo4j数据集和五个代码专用语言模型上，RAS在n=5时将查询执行错误率降低了41%~50%，优于IS的32%~38%。执行错误不仅是失败信号，更是可利用的反馈，围绕其组织推理期计算比单纯扩大独立样本更高效地提升可执行性。
原文摘要 · Abstract (English)
Inference-time scaling can reduce errors in structured query generation, but methods to allocate the compute for query code generation remains underexplored. We study Text2Cypher, where language models generate Cypher queries that execute against property graph databases. Non-executable queries constitute a distinct syntactic failure separate from semantic inaccuracy: a syntax error triggers a system-generated error message from the database. These error messages are typically discarded at inference time rather than leveraged through in-context learning (ICL). We compare two inference methods: Independent Scaling (IS), which performs memoryless resampling, and Reflection-Augmented Scaling (RAS), which conditions each new attempt on prior execution feedback via ICL. Across three Neo4j datasets and five code-specialized language models, RAS reduces the Query Execution Error Rate by 41--50% at n{=}5, outperforming IS at 32--38%. Execution errors are not merely failures to discard but actionable feedback, and structuring inference-time compute around them is a more efficient path to executability than scaling independent samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。