arXiv:2411.02948cs.DBcs.CL2024-11被引 18

让翻译SQL的模型自己检查结果,通过数据解释提升准确率。

Grounding Natural Language to SQL Translation with Data-Based Self-Explanations

  • 用查询结果的数据解释作为反馈,让模型迭代优化SQL生成。
  • 在Spider数据集上,最优模型准确率提升至81.6%,较原模型提高3.2%。
  • 生成的自然语言解释帮助用户理解结果,增强系统可解释性。

自然语言数据库接口使非技术用户能用自然语言与数据交互。现有先进方法通常采用端到端神经序列转换或大规模语言模型进行自然语言到SQL(NL2SQL)翻译,但这类模型如人类一样,首次生成的SQL未必最优。本文提出CycleSQL,一种用于端到端模型的迭代框架,使其可通过自我评估生成最佳输出。核心思想是引入基于数据的自然语言查询结果解释作为自提供反馈,利用反馈循环验证并逐步改进翻译正确性,从而提升整体准确率。大量实验在五个常用基准上对七种现有模型应用该框架,结果表明:1)反馈机制能持续提升模型性能,尤其在应用到RESDSQL时,在Spider验证集上达到82.0%(+2.6%),测试集上达81.6%(+3.2%);2)生成的自然语言解释能为用户提供洞察,辅助理解翻译结果,显著提升NL2SQL系统的可解释性。

原文摘要 · Abstract (English)

Natural Language Interfaces for Databases empower non-technical users to interact with data using natural language (NL). Advanced approaches, utilizing either neural sequence-to-sequence or more recent sophisticated large-scale language models, typically implement NL to SQL (NL2SQL) translation in an end-to-end fashion. However, like humans, these end-to-end translation models may not always generate the best SQL output on their first try. In this paper, we propose CycleSQL, an iterative framework designed for end-to-end translation models to autonomously generate the best output through self-evaluation. The main idea of CycleSQL is to introduce data-grounded NL explanations of query results as self-provided feedback, and use the feedback to validate the correctness of the translation iteratively, hence improving the overall translation accuracy. Extensive experiments, including quantitative and qualitative evaluations, are conducted to study CycleSQL by applying it to seven existing translation models on five widely used benchmarks. The results show that 1) the feedback loop introduced in CycleSQL can consistently improve the performance of existing models, and in particular, by applying CycleSQL to RESDSQL, obtains a translation accuracy of 82.0% (+2.6%) on the validation set, and 81.6% (+3.2%) on the test set of Spider benchmark; 2) the generated NL explanations can also provide insightful information for users, aiding in the comprehension of translation results and consequently enhancing the interpretability of NL2SQL translation.

NL2SQL自解释迭代优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。