自动生成证据提升文本转SQL的准确率与实用性
SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation
- 通过分析数据库模式、描述文件和值自动提取相关证据
- 在无现成证据情况下，准确率显著提升，部分超过原数据集表现
- 适合追求真实场景落地的文本转SQL研究者与开发者
文本转SQL使非专家可通过自然语言查询数据库。然而现有研究依赖BIRD数据集，该数据集假设证据已提供，这要求用户具备专业知识，违背了文本转SQL的核心目标。此外，人工标注的证据存在缺失或错误，影响模型性能。为此，我们提出SEED（证据提取与领域知识生成系统），通过分析数据库模式、描述文件和实际值，自动构建相关证据。我们在BIRD和Spider数据集上评估，结果表明：在无证据条件下，SEED显著提升SQL生成准确率，某些情况下甚至优于提供原始证据的设置。实验说明，由SEED生成的证据不仅弥合了研究与实际应用的差距，还增强了模型的适应性与鲁棒性。代码已开源：https://github.com/felix01189/SEED。
原文摘要 · Abstract (English)
Text-to-SQL enables non-experts to retrieve data from databases by converting natural language queries into SQL. However, state-of-the-art text-to-SQL studies rely on the BIRD dataset, which assumes that evidence is provided along with questions. Although BIRD facilitates research advancements, it assumes that users have expertise and domain knowledge, contradicting the fundamental goal of text-to-SQL. In addition, human-generated evidence in BIRD contains defects, including missing or erroneous evidence, which affects model performance. To address this issue, we propose SEED (System for Evidence Extraction and Domain knowledge generation), an approach that automatically generates evidence to improve performance and practical usability in real-world scenarios. SEED systematically analyzes database schema, description files, and values to extract relevant information. We evaluated SEED on BIRD and Spider, demonstrating that it significantly improves SQL generation accuracy in the no-evidence scenario, and in some cases, even outperforms the setting where BIRD evidence is provided. Our results highlight that SEED-generated evidence not only bridges the gap between research and real-world deployment but also improves the adaptability and robustness of text-to-SQL models. Our code is available at https://github.com/felix01189/SEED
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。