通过渐进奖励提升文本转SQL的多轮纠错能力
Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards

- 用结构诊断树分析语句层级,生成每步修正反馈
- 结合结构与词法对齐,设计逐步优化的奖励函数
- 适合需要精细调试的复杂数据库查询任务
强化学习在提升大语言模型的文本转SQL生成能力方面展现出潜力,但现有方法通常仅基于单次SQL状态定义一次性奖励,难以指导多轮迭代修正,且无法充分捕捉逐步优化过程。本文提出Progress-SQL,一种支持多轮交互的强化学习框架,引入基于Oracle的诊断树(ODT),将SQL查询抽象为条款级结构特征,并提供下一回合的修正反馈。为获得密集且稳健的奖励信号,结合ODT结构对齐与词法对齐,构建衡量从初始到最终SQL改进程度的渐进式奖励。此外,还加入进度延迟奖励以鼓励早期正确性,以及执行状态奖励以促进对无效SQL的恢复。在BIRD、Spider及Spider鲁棒性变体上的实验表明,该方法在主评测与鲁棒性评测中均持续提升文本转SQL性能。
原文摘要 · Abstract (English)
Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to capture the improvement of multi-turn SQL refinement. In this paper, we propose Progress-SQL, a multi-turn reinforcement learning framework with progressive rewards for Text-to-SQL. Our approach introduces an Oracle-guided Diagnostic Tree (ODT), which abstracts SQL queries into clause-level structural profiles and produces diagnostic feedback for next-turn refinement. To provide dense and robust reward signals, we combine ODT-based structural alignment with lexical alignment and define a progressive reward that measures the improvement from the initial SQL to the final SQL. We further incorporate a progression latency reward that favors earlier correctness and an execution status reward that encourages recovery from the invalid SQL. Experiments on BIRD, Spider, and Spider robustness variants demonstrate that our method consistently improves Text-to-SQL performance across both primary and robustness evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。