用合成数据提升开源大模型的文本转SQL能力
SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs
- 通过语法约束和逆向翻译保证数据可靠性
- 85.7%准确率刷新开源模型在Spider上的表现
- 适合想改进文本转SQL能力的研究者
大型语言模型（LLMs）在文本转SQL任务中展现出巨大潜力，但现有开源模型与闭源模型之间仍存在显著性能差距。本文提出SQLForge，一种合成可靠且多样数据的新方法，以增强LLMs的文本转SQL推理能力。通过SQL语法约束和SQL-to-question逆向翻译，确保数据在结构和语义层面的逻辑一致性；同时提出SQL模板增强与迭代数据域探索机制，提升数据多样性。基于扩充后的数据，我们对多种架构和参数规模的开源模型进行微调，形成名为SQLForge-LM的模型系列。该系列在广泛使用的Spider和BIRD基准上达到当前开源模型的最优表现，其中在Spider Dev集上实现85.7%的精确匹配（EX）准确率，在BIRD Dev集上达到59.8%，显著缩小了与闭源方法的性能差距。
原文摘要 · Abstract (English)
Large Language models (LLMs) have demonstrated significant potential in text-to-SQL reasoning tasks, yet a substantial performance gap persists between existing open-source models and their closed-source counterparts. In this paper, we introduce SQLForge, a novel approach for synthesizing reliable and diverse data to enhance text-to-SQL reasoning in LLMs. We improve data reliability through SQL syntax constraints and SQL-to-question reverse translation, ensuring data logic at both structural and semantic levels. We also propose an SQL template enrichment and iterative data domain exploration mechanism to boost data diversity. Building on the augmented data, we fine-tune a variety of open-source models with different architectures and parameter sizes, resulting in a family of models termed SQLForge-LM. SQLForge-LM achieves the state-of-the-art performance on the widely recognized Spider and BIRD benchmarks among the open-source models. Specifically, SQLForge-LM achieves EX accuracy of 85.7% on Spider Dev and 59.8% on BIRD Dev, significantly narrowing the performance gap with closed-source methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。