用图匹配方法实现无执行强化学习，提升文本转SQL效率与准确率
Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward
- 基于SQL图结构设计奖励模型，避免重复数据库调用
- 在Spider和BIRD数据集上超越现有方法，准确率显著提升
- 支持分步奖励机制，提升生成SQL的可读性与正确性
强化学习（RL）被广泛用于提升大语言模型在文本转SQL任务上的表现。然而，现有方法多依赖执行反馈或基于LLM的Bradley-Terry奖励模型：前者因频繁数据库调用导致高延迟，后者则带来巨大GPU显存开销，严重制约了RL管道的效率与可扩展性。为此，我们提出Graph-Reward-SQL，一种基于图匹配的新型奖励模型框架，采用GMNScore作为奖励信号。通过构建SQL图表示，在大幅降低时间成本和显存占用的同时提供精确奖励。在此基础上，进一步提出StepRTM，一种针对公共表表达式（CTE）子查询的分步奖励机制，实现对中间生成过程的监督，提升最终SQL的功能正确性与可读性。在Spider和BIRD等标准基准上的大量对比与消融实验表明，本方法持续优于现有奖励模型。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has been widely adopted to enhance the performance of large language models (LLMs) on Text-to-SQL tasks. However, existing methods often rely on execution-based or LLM-based Bradley-Terry reward models. The former suffers from high execution latency caused by repeated database calls, whereas the latter imposes substantial GPU memory overhead, both of which significantly hinder the efficiency and scalability of RL pipelines. To this end, we propose a novel reward model framework for RL-based Text-to-SQL named Graph-Reward-SQL, which employs the GMNScore outcome reward model. We leverage SQL graph representations to provide accurate reward signals while significantly reducing time cost and GPU memory usage. Building on this foundation, we further introduce StepRTM, a stepwise reward model that provides intermediate supervision over Common Table Expression (CTE) subqueries. This encourages both functional correctness and readability of SQL. Extensive comparative and ablation experiments on standard benchmarks, including Spider and BIRD, demonstrate that our method consistently outperforms existing reward models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。