用语法树引导分解,让小模型也能达到GPT-4的SQL生成水平
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
- 通过语法树引导搜索生成可验证的查询分解路径
- 70亿参数模型在基准上媲美GPT-4的准确率
- 适合资源受限下追求透明、可复现NL2SQL的开发者
自然语言转SQL(NL2SQL)旨在将自然语言查询转化为可执行的SQL语句,使非专业用户能直观访问数据库。尽管基于大型私有大模型(如GPT-4)的方法已达到顶尖性能,但面临开放性不足与推理时计算成本高昂的问题。为此,我们探索在资源受限环境下提升小型公开大模型在NL2SQL上的表现。实验发现任务分解具有潜力,但难以有效引导模型完成。据此提出LearNAT框架:一是引入基于抽象语法树(AST)引导的分解合成过程,结合剪枝策略生成可验证且高效的分解;二是采用边界感知强化学习,在多步推理中实现比标准DPO更精细的偏好优化。在多个基准数据集上的大量实验表明,LearNAT显著提升了小模型性能,仅用70亿参数模型即达到与GPT-4相当的效果。结果验证了可验证分解与细粒度偏好学习对推动NL2SQL向开放性、透明性与高效性发展的有效性。代码已开源:https://github.com/MrBlankness/LearNAT。
原文摘要 · Abstract (English)
Natural Language to SQL (NL2SQL) aims to translate natural language queries into executable SQL statements, offering non-expert users intuitive access to databases. While recent approaches leveraging large-scale private LLMs such as GPT-4 have achieved state-of-the-art results, they face two critical challenges: the lack of openness and reproducibility, and the prohibitive computational cost of test-time scaling. To address these issues, we explore improving the model-level performance of small-scale public LLMs in NL2SQL under resource-constrained settings. Our exploratory experiments reveal the potential of task decomposition for enhancing NL2SQL performance, but also highlight the difficulty of enabling LLMs to decompose queries effectively. Motivated by these findings, we propose LearNAT, a novel framework designed to enhance decomposition capabilities of LLM. LearNAT introduces (1) a Decomposition Synthesis Procedure, which leverages AST-guided search with pruning strategies to generate verifiable and efficient decompositions, and (2) Margin-Aware Reinforcement Learning, which provides fine-grained preference optimization for multi-step reasoning beyond standard DPO. Extensive experiments on benchmark datasets demonstrate that LearNAT significantly improves the performance of small-scale LLMs, achieving results comparable to GPT-4 with only a 7B parameter model. These results validate the effectiveness of verifiable decomposition and fine-grained preference learning in advancing NL2SQL towards openness, transparency, and efficiency. Our code is publicly available at https://github.com/MrBlankness/LearNAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。