arXiv:2502.13550cs.CL2025-02ACL被引 30

让大模型像解题一样逐步推理生成SQL,准确率提升31.6%。

STaR-SQL: Self-Taught Reasoner for Text-to-SQL

  • 用思维链引导模型分步推理生成SQL
  • 在Spider数据集上达86.6%执行准确率
  • 适合需要高精度结构化查询的开发者

生成逐步推理过程已被证明能提升大语言模型在复杂推理任务上的表现。然而,将其应用于结构化任务如文本转SQL仍鲜有探索。本文提出自教学推理器STaR-SQL,将SQL生成重构为基于推理的过程。该方法引导大模型生成详细推理步骤,并在正确结果的推理路径上进行微调。与传统方法不同,STaR-SQL在测试阶段额外投入计算用于推理,使模型成为自发推理者而非仅依赖提示的代理。为进一步扩展推理规模,引入结果监督奖励模型(ORM)作为验证器,提升SQL生成准确率。在挑战性Spider基准上的实验表明,STaR-SQL显著提升性能,执行准确率达86.6%,比少样本基线高出31.6%,比直接预测答案的微调基线高出18.0%。此外,其效果优于使用更强大但闭源模型如GPT-4的代理式提示方法。这些结果凸显了推理增强训练在结构化任务中的潜力,并为自改进推理模型在文本转SQL及其他领域的扩展铺平道路。

原文摘要 · Abstract (English)

Generating step-by-step "chain-of-thought" rationales has proven effective for improving the performance of large language models on complex reasoning tasks. However, applying such techniques to structured tasks, such as text-to-SQL, remains largely unexplored. In this paper, we introduce Self-Taught Reasoner for text-to-SQL (STaR-SQL), a novel approach that reframes SQL query generation as a reasoning-driven process. Our method prompts the LLM to produce detailed reasoning steps for SQL queries and fine-tunes it on rationales that lead to correct outcomes. Unlike traditional methods, STaR-SQL dedicates additional test-time computation to reasoning, thereby positioning LLMs as spontaneous reasoners rather than mere prompt-based agents. To further scale the inference process, we incorporate an outcome-supervised reward model (ORM) as a verifier, which enhances SQL query accuracy. Experimental results on the challenging Spider benchmark demonstrate that STaR-SQL significantly improves text-to-SQL performance, achieving an execution accuracy of 86.6%. This surpasses a few-shot baseline by 31.6% and a baseline fine-tuned to predict answers directly by 18.0%. Additionally, STaR-SQL outperforms agent-like prompting methods that leverage more powerful yet closed-source models such as GPT-4. These findings underscore the potential of reasoning-augmented training for structured tasks and open the door to extending self-improving reasoning models to text-to-SQL generation and beyond.

文本转SQL思维链大模型推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。