通过多轮工具调用与动态反馈,提升文本转SQL模型的准确率。
MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQL
- 在每步推理中融入数据库执行反馈,实现上下文感知的逐步优化。
- 在BIRD Dev上达64.4%准确率,SPIDER Dev上84.6%执行准确率。
- 适合需要高精度文本转SQL且关注推理过程可控性的研究者。
随着大语言模型在文本转SQL任务中的广泛应用,强化学习已成为提升性能的常用方法。现有方法主要依赖静态执行反馈,限制了实时纠错能力。通过引入多轮工具调用与动态反馈,可显著提升模型的适应性与鲁棒性。为此,我们提出MTIR-SQL,一种面向文本转SQL的多轮工具集成推理强化学习框架。该方法引入执行感知的多轮推理范式,在每一步推理中无缝整合数据库执行反馈,实现上下文敏感的查询生成与渐进式优化。框架将GRPO算法扩展至复杂多轮交互场景。针对MTIR训练不稳定性及模型分布偏离初始模型的问题,我们改进GRPO算法,加入轨迹过滤机制并移除KL损失约束。实验结果表明,参数量为4B的MTIR-SQL在BIRD Dev上达到64.4%准确率,在SPIDER Dev上实现84.6%执行准确率,显著优于现有方法。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly used in Text-to-SQL tasks, Reinforcement Learning (RL) has become a common method for improving performance. Existing methods primarily rely on static execution feedback, which restricts real-time error correction. However, integrating multi-turn tool invocation along with dynamic feedback could significantly improve adaptability and robustness, ultimately enhancing model performance. To address these issues, we propose MTIR-SQL, an innovative Multi-turn Tool-Integrated Reasoning reinforcement learning framework for Text-to-SQL. Our approach introduces an execution-aware multi-turn reasoning paradigm that seamlessly incorporates database execution feedback at each reasoning step, enabling context-sensitive query generation and progressive refinement throughout the reasoning process. The framework extends the GRPO algorithm to accommodate complex multi-turn interaction scenarios. Considering the training instability characteristics of MTIR and the potential for significant Deviation of model distribution from the initial model, we enhance the GRPO algorithm by adding a trajectory filtering mechanism and removing KL loss constraints. Experimental results demonstrate that MTIR-SQL, with 4B parameters, achieves \textbf{64.4}\% accuracy in the BIRD Dev and 84.6% execution accuracy in the SPIDER Dev, significantly outperforming existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。