通过部分匹配奖励与语言强化学习,提升复杂数据库下的文本转SQL准确率。
PaVeRL-SQL: Text-to-SQL via Partial-Match Rewards and Verbal Reinforcement Learning
- 用部分匹配奖励和语言强化学习驱动模型自优化
- 在Spider2.0-SQLite上比当前最佳高出7.4%执行准确率
- 适合工业级复杂查询场景,支持多种SQL方言
文本转SQL模型可让用户通过自然语言提问生成可执行的SQL语句。尽管在简单数据库和问题上取得进展,现有方法在包含领域业务逻辑的工业级复杂问题上仍存在执行准确率低的问题。本文提出PaVeRL-SQL框架,结合部分匹配奖励与语言强化学习,推动推理语言模型在文本转SQL任务上的自我改进。采用两种实用管道:(1) 基于群组自评估的语言模型上下文学习框架(言语强化学习),使用开放与闭源大模型作为主干;(2) 采用小规模主干模型OmniSQL-7B的思维链强化学习管道,通过定制奖励函数和两阶段强化学习训练。两个管道在Spider、Spider 2.0和BIRD等主流基准上均达到最先进水平。在工业级Spider2.0-SQLite基准上,言语强化学习管道执行准确率比现有最佳高7.4%,思维链管道高1.4%。混合多种SQL方言的强化学习训练带来三倍性能提升,尤其对训练数据少的方言效果显著。整体上,PaVeRL-SQL在真实工业约束下实现可靠且领先的文本转SQL能力。代码已开源:https://github.com/PaVeRL-SQL/PaVeRL-SQL。
原文摘要 · Abstract (English)
Text-to-SQL models allow users to interact with a database more easily by generating executable SQL statements from natural-language questions. Despite recent successes on simpler databases and questions, current Text-to-SQL methods still suffer from low execution accuracy on industry-scale databases and complex questions involving domain-specific business logic. We present \emph{PaVeRL-SQL}, a framework that combines \emph{Partial-Match Rewards} and \emph{Verbal Reinforcement Learning} to drive self-improvement in reasoning language models (RLMs) for Text-to-SQL. To handle practical use cases, we adopt two pipelines: (1) a newly designed in-context learning framework with group self-evaluation (verbal-RL), using capable open- and closed-source large language models (LLMs) as backbones; and (2) a chain-of-thought (CoT) RL pipeline with a small backbone model (OmniSQL-7B) trained with a specially designed reward function and two-stage RL. These pipelines achieve state-of-the-art (SOTA) results on popular Text-to-SQL benchmarks -- Spider, Spider 2.0, and BIRD. For the industrial-level Spider2.0-SQLite benchmark, the verbal-RL pipeline achieves an execution accuracy 7.4\% higher than SOTA, and the CoT pipeline is 1.4\% higher. RL training with mixed SQL dialects yields strong, threefold gains, particularly for dialects with limited training data. Overall, \emph{PaVeRL-SQL} delivers reliable, SOTA Text-to-SQL under realistic industrial constraints. The code is available at https://github.com/PaVeRL-SQL/PaVeRL-SQL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。