用细粒度条款批评提升文本转SQL的纠错能力
SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Critic
- 按子句级别分析错误，精准定位语法与语义问题
- 在BIRD和Spider数据集上显著提升SQL生成准确率
- 支持自动构建低成本标注数据集，适合大模型训练
基于大语言模型的文本转SQL系统中，现有修正方法效果有限，常在自纠正过程中引入新错误，且难以检测和修正语义偏差。为此，我们首次提出条款级批评生成任务及基准SQLCriticBench，实现对语法和语义错误的细粒度定位。我们还提出一种改进的直接偏好优化（DPO）方法，根据优选与次优批评之间的条款级不一致动态调整β系数。同时设计了一种自动化数据集构建流程，可低成本、大规模标注条款级批评。实验表明，SQLCritic模型在BIRD和Spider数据集上的SQL准确率显著提升，且在SQLCriticBench上的表现优于现有模型。
原文摘要 · Abstract (English)
Existing refinement methods in LLM-based Text-to-SQL systems exhibit limited effectiveness. They often introduce new errors during the self-correction process and fail to detect and correct semantic inaccuracies. To address these gaps, we first introduce a clause-wise critique generation task along with a benchmark, SQLCriticBench, which performs fine-grained error localization including both syntax and semantic errors at the clause level. Furthermore, we introduce a variant of DPO for training our SQLCritic model, where the $β$ coefficient is adaptively changed according to the clause-level inconsistencies between the preferred and dispreferred critiques. We also propose an automatically training dataset curation pipeline which annotate clause-wise critique at scale in a cost-effective way. Experiments demonstrate that the SQLCritic model significantly improves SQL accuracy on the BIRD and Spider datasets, and the results on SQLCriticBench further reveals its superior critique capabilities compared to existing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。