通过执行反馈精炼策略,提升文本转SQL的多轮对话准确率。
SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

- 基于执行结果重评动作,实现局部责任分配
- 在BIRD和Spider数据集上分别达76.56%和89.92%准确率
- 仅用轻量级执行奖励即可筛选高质量候选答案
当前文本转SQL系统越来越依赖多轮交互、执行反馈与强化学习。但多数方法仅将执行正确性作为轨迹级奖励,难以定位导致成功或失败的具体SQL决策。本文提出SERL-SQL,一种面向多轮文本转SQL智能体的选择性执行引导强化学习框架。该框架采样在线策略的SQL交互轨迹,并使用仅用于训练的教师模型,结合执行反馈对学生动作进行重新评分。由此产生的师生似然差距转化为有界掩码权重,仅对SQL和工具操作标记的梯度奖励优势进行重加权。这样,任务奖励保持优化方向,而执行回溯提供局部信用分配。在BIRD、Spider及跨域基准上的实验表明,SERL-SQL表现优异,在BIRD-Dev上达到76.56%执行准确率,在Spider-Test上达89.92%。此外,基于奖励的选择策略接近最优的Best-of-N上界,且持续优于基于一致性的选择策略,说明SERL-SQL能生成可被轻量级执行奖励可靠识别的高质量候选答案。代码将开源于https://github.com/Ffunkytao/SERL-SQL。
原文摘要 · Abstract (English)
Recent Text-to-SQL systems increasingly rely on multi-turn interaction, execution feedback, and reinforcement learning. However, most existing methods use execution correctness only as a trajectory-level reward, which provides limited guidance for identifying the SQL decisions responsible for success or failure. We propose SERL-SQL, a selective execution-grounded reinforcement learning framework for multi-turn Text-to-SQL agents. SERL-SQL samples on-policy SQL interaction trajectories and uses a training-only teacher to re-score student actions with execution feedback. The resulting teacher--student likelihood gap is converted into bounded, masked weights that reweight GRPO advantages only on SQL and tool-action tokens. In this way, task rewards preserve the optimization direction, while execution hindsight provides localized credit assignment. Experiments on BIRD, Spider, and cross-domain benchmarks show that SERL-SQL achieves competitive performance, reaching 76.56% execution accuracy on BIRD-Dev and 89.92% on Spider-Test. Moreover, our reward-based selection strategy closely approaches the oracle Best-of-N upper bound and consistently outperforms consistency-based selection, showing that SERL-SQL produces high-quality candidates that can be reliably identified by lightweight execution-grounded rewards. Our code will be released at https://github.com/Ffunkytao/SERL-SQL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。