用SQL质量评估提升大模型文本转SQL效果，无需额外工具
Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement
- 通过评估生成SQL的语法和语义正确性，构建反馈循环优化模型
- 在BIRD数据集上达到与GPT4、T5相当的执行准确率和有效率
- 适合希望简化微调流程的NLP工程师或数据库应用开发者
文本转SQL使非专业用户能通过自然语言查询关系型数据库获取信息。尽管近期基于大语言模型（如GPT、T5）的方法在大规模基准BIRD上表现优异，但当前最先进的模型通常需要开发辅助工具（如SQL分类器）才能取得高精度。本文提出一种新方法，仅依赖SQL质量评估即可提升基于大模型的文本转SQL性能。该方法建立了一个基于预设标准与实际数据库响应的SQL质量评价机制，形成反馈循环，持续优化模型输出的语法正确性和语义准确性。在BIRD基准上进行的全面验证显示，该方法在不同难度级别的文本转SQL任务中，均取得了与SOTA模型（如GPT4、T5）相当的执行准确率（EX）和有效效率得分（VES）。
原文摘要 · Abstract (English)
Text-to-SQLs enables non-expert users to effortlessly retrieve desired information from relational databases using natural language queries. While recent advancements, particularly with Large Language Models (LLMs) like GPT and T5, have shown impressive performance on large-scale benchmarks such as BIRD, current state-of-the-art (SOTA) LLM-based Text-to-SQLs models often require significant efforts to develop auxiliary tools like SQL classifiers to achieve high performance. This paper proposed a novel approach that only needs SQL Quality Measurement to enhance LLMs-based Text-to-SQLs performance. It establishes a SQL quality evaluation mechanism to assess the generated SQL queries against predefined criteria and actual database responses. This feedback loop enables continuous learning and refinement of model outputs based on both syntactic correctness and semantic accuracy. The proposed method undergoes comprehensive validation on the BIRD benchmark, assessing Execution Accuracy (EX) and Valid Efficiency Score (VES) across various Text-to-SQLs difficulty levels. Experimental results reveal competitive performance in both EX and VES compared to SOTA models like GPT4 and T5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。