arXiv:2505.12768cs.CL2025-05ACL被引 12

让大模型边生成SQL边执行反馈,实时修正错误

ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL

  • 生成时嵌入数据库执行,动态调整推理路径
  • 7B模型在Spider上达88.8%,比基线高2.7%
  • 支持探索性推理,效率提升51.9%

在文本转SQL任务中,执行反馈对引导大语言模型准确推理和生成可靠SQL至关重要。然而,现有方法仅将执行反馈作为事后校正或选择信号,未能将其融入生成过程,导致难以及时发现并修正推理错误,影响查询准确率与鲁棒性。为此,我们提出ReEx-SQL(基于执行感知强化学习的推理框架),使模型在解码过程中与数据库交互,并根据执行反馈动态调整推理。该框架引入执行感知推理范式,将中间SQL执行结果嵌入推理路径,实现上下文敏感的修正。通过带标记的结构化提示和分步展开策略,将执行反馈融入每一步生成。为监督策略学习,设计包含探索奖励的复合奖励函数,显式鼓励有效数据库交互。此外,采用树状解码策略支持探索性推理,实现推理路径的动态扩展。ReEx-SQL在7B规模下,于Spider上达到88.8%、BIRD上达64.9%,分别超越标准推理基线2.7%和2.6%;在Spider-Realistic上表现稳健,达85.2%。其树状解码相较线性解码效率更高,于BIRD开发集上推理时间减少51.9%。

原文摘要 · Abstract (English)

In Text-to-SQL, execution feedback is essential for guiding large language models (LLMs) to reason accurately and generate reliable SQL queries. However, existing methods treat execution feedback solely as a post-hoc signal for correction or selection, failing to integrate it into the generation process. This limitation hinders their ability to address reasoning errors as they occur, ultimately reducing query accuracy and robustness. To address this issue, we propose ReEx-SQL (Reasoning with Execution-Aware Reinforcement Learning), a framework for Text-to-SQL that enables models to interact with the database during decoding and dynamically adjust their reasoning based on execution feedback. ReEx-SQL introduces an execution-aware reasoning paradigm that interleaves intermediate SQL execution into reasoning paths, facilitating context-sensitive revisions. It achieves this through structured prompts with markup tags and a stepwise rollout strategy that integrates execution feedback into each stage of generation. To supervise policy learning, we develop a composite reward function that includes an exploration reward, explicitly encouraging effective database interaction. Additionally, ReEx-SQL adopts a tree-based decoding strategy to support exploratory reasoning, enabling dynamic expansion of alternative reasoning paths. Notably, ReEx-SQL achieves 88.8% on Spider and 64.9% on BIRD at the 7B scale, surpassing the standard reasoning baseline by 2.7% and 2.6%, respectively. It also shows robustness, achieving 85.2% on Spider-Realistic with leading performance. In addition, its tree-structured decoding improves efficiency and performance over linear decoding, reducing inference time by 51.9% on the BIRD development set.

文本转SQL强化学习执行反馈大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。