arXiv:2608.27796cs.AI2026-08

用两阶段强化学习让文本转SQL更鲁棒,错误可迭代修复。

ReToolSQL: Agentic Reinforcement Learning for Robust Text-to-SQL

论文配图:ReToolSQL: Agentic Reinforcement Learning for Robust Text-to-SQL
图 1 · 摘自论文原文
  • 先用监督学习训练模型生成可验证的推理路径,再通过多轮工具使用强化学习优化决策。
  • 在BIRD-SQL上单次通过准确率达74.32%,自洽时达74.77%,位居榜首。
  • 无需人工标注,仅用一个310亿参数模型即可实现企业级可靠性能。

近期研究显示,基于执行反馈的强化学习能显著提升文本转SQL性能,使小模型达到甚至超越大模型水平。然而,多数方法将SQL生成视为单轮任务,限制了模型通过迭代修正错误的能力。本文提出ReToolSQL,一种两阶段文本转SQL训练框架,结合(i)在拒绝采样推理轨迹上的监督预热,以及(ii)基于多轮工具使用轨迹的代理式强化微调(RFT)。关键洞察是两阶段在互补维度起作用:监督微调(SFT)利用已验证的教师轨迹扩展可解问题范围(提升最困难案例的pass@k覆盖率),而RFT则将此能力转化为更高单次通过准确率,教会模型何时验证、检索何种证据,以及如何根据执行反馈修复错误的SQL。应用于Gemma 4指令微调模型(310亿参数),仅使用RFT即在BIRD-SQL开发集上达到73.66%执行准确率(自洽时为74.12%)。从SFT检查点初始化RFT(SFT→RFT)得到最强模型,单次通过准确率为74.32%,自洽时达74.77%。截至撰写时,该结果在BIRD单模型开发集排行榜中排名第一。该方法采用以执行正确性为核心的复合奖励,除基准数据集外无需人工标注,且在单一密集型310亿参数模型内完成,表明设计良好的SFT→RFT流程是实现稳健企业级文本转SQL的可行路径。

原文摘要 · Abstract (English)

Recent work has shown that reinforcement learning from execution feedback can substantially improve text-to-SQL performance, often enabling smaller models to match or exceed much larger systems. However, most existing approaches treat SQL generation as a single-turn task, limiting the model's ability to recover from errors through iterative refinement. We present ReToolSQL, a two-stage training framework for text-to-SQL that combines (i) a supervised warm-start on rejection-sampled reasoning traces with (ii) agentic reinforcement fine-tuning (RFT) over multi-turn tool-use trajectories. The key insight is that the two stages act on complementary axes, the supervised fine-tuning (SFT) on verified privileged-teacher traces expands the set of solvable questions (raising pass@k coverage on the hardest cases), while RFT converts that expanded capability into higher single-pass accuracy by teaching the model when to verify, what evidence to retrieve, and how to repair faulty SQL from execution feedback. Applied to Gemma 4 instruction-tuned (31B), RFT alone achieves 73.66% execution accuracy (EX) on the BIRD-SQL development benchmark (74.12% EX with self-consistency). Initializing RFT from the SFT checkpoint (SFT$\to$RFT) yields our strongest model at 74.32% EX single-pass and 74.77% EX with self-consistency. At the time of writing, this ranked first on the BIRD single-model development-set leaderboard. The approach uses composite rewards anchored on execution correctness, requires no human annotation beyond the benchmark itself, and operates within a single dense 31B model, showing that a properly designed SFT$\to$RFT pipeline over tool-use trajectories is a practical path toward robust enterprise-grade text-to-SQL.

文本转SQL强化学习多轮推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。