用小模型实现高效文本转SQL，准确率逼近大模型。
SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
- 用合成数据+强化学习微调小模型，提升逻辑推理能力。
- 0.5B模型达56.87%执行准确率，1.5B模型达67.08%。
- 适合边缘部署，推理快，代码开源可复现。
大语言模型在文本转SQL任务中表现优异，而参数量在0.5B至1.5B的小语言模型（SLMs）因逻辑推理能力有限，目前表现较差。然而，小模型在推理速度和边缘部署方面具有天然优势。为探索其在文本转SQL中的潜力，我们利用近期后训练技术，基于开源SynSQL-2.5M数据集构建了两个衍生数据集：用于生成SQL的SynSQL-Think-916K，以及用于合并修正的SynSQL-Merge-Think-310K。随后对SLM进行监督微调与基于强化学习的后训练，并采用纠正性自一致性推理。实验表明，所提方法SLM-SQL有效且具备良好泛化能力。在BIRD开发集上，五种模型平均提升31.4分。其中，0.5B模型达到56.87%执行准确率（EX），1.5B模型达67.08%。相关数据集、模型与代码将发布于GitHub：https://github.com/CycloneBoy/slm_sql。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong performance in translating natural language questions into SQL queries (Text-to-SQL). In contrast, small language models (SLMs) ranging from 0.5B to 1.5B parameters currently underperform on Text-to-SQL tasks due to their limited logical reasoning capabilities. However, SLMs offer inherent advantages in inference speed and suitability for edge deployment. To explore their potential in Text-to-SQL applications, we leverage recent advancements in post-training techniques. Specifically, we used the open-source SynSQL-2.5M dataset to construct two derived datasets: SynSQL-Think-916K for SQL generation and SynSQL-Merge-Think-310K for SQL merge revision. We then applied supervised fine-tuning and reinforcement learning-based post-training to the SLM, followed by inference using a corrective self-consistency approach. Experimental results validate the effectiveness and generalizability of our method, SLM-SQL. On the BIRD development set, the five evaluated models achieved an average improvement of 31.4 points. Notably, the 0.5B model reached 56.87\% execution accuracy (EX), while the 1.5B model achieved 67.08\% EX. We will release our dataset, model, and code to github: https://github.com/CycloneBoy/slm_sql.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。