首个阿拉伯语文本转SQL数据集,提升非技术用户数据库对话能力
Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabic
- 构建首个阿拉伯语跨领域上下文相关文本转SQL数据集Ar-SParC
- GAT纠正器在零样本和上下文学习下分别提升1.9%和1.72%执行准确率
- 适用于自然语言查询数据库的阿拉伯语研究者与开发者
近年来,跨领域、上下文相关的文本转SQL任务受到广泛关注,使无SQL知识的用户能通过自然语言与数据库交互。然而,现有数据集与研究主要集中在英语,少数涉及中文,尚未有工作针对阿拉伯语。本文提出首个阿拉伯语跨领域、上下文相关的文本转SQL数据集Ar-SParC,包含3,450个相互关联的问题序列,平均每序列约3个问题,总计10,225个问题及其对应SQL查询。我们在Ar-SParC上使用GPT-3.5-turbo和GPT-4.5-turbo进行40次实验,采用10种提示工程技巧(含4种问题表示方法和6种上下文学习技术)。此外,提出新型GAT纠正器,在零样本设置下平均提升1.9%执行准确率(EX)和1.9%交互准确率(IX),在上下文学习设置下分别提升1.72%(EX)和0.92%(IX)。最后通过消融实验分析了GAT纠正器优于先前GAT验证器的原因,尤其对阿拉伯语更有效。
原文摘要 · Abstract (English)
In recent years, the task of cross-domain, context-dependent text-to-SQL has received significant attention. Enables users with no prior knowledge of SQL to have a conversation with databases using natural language. However, most of the available datasets and research have been conducted in English, along with some work in Chinese. To this date, no effort has been made to address this task in the Arabic language. In this paper, we introduce Ar-SParC, the first Arabic cross-domain, context-dependent text-to-SQL dataset. The dataset consists of 3,450 sequences of interrelated questions, each sequence containing an average of approximately three questions, which results in a total of 10225 questions along with their corresponding SQL queries. We conducted 40 experiments on the Ar-SParC dataset using two large language models, GPT-3.5-turbo and GPT-4.5-turbo, applying 10 different prompt engineering techniques, including four question representation methods and six in-context learning techniques. Furthermore, we developed a novel approach named GAT corrector, which enhanced the performance across all 40 experiments, yielding an average improvement of 1.9% in execution accuracy (EX) and 1.9% in interaction accuracy (IX) under zero-shot settings, and an average increase of 1.72% EX and 0.92% IX under in-context learning settings. Finally, we conducted an ablation study with two more experiments to explain why the GAT corrector outperformed the previous GAT verifier technique, particularly for the Arabic language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。