通过显式建模SQL错误类型,提升文本转SQL的纠错能力。
ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinement
- 用结构化特征和专用错误标记显式建模语义错误。
- 在Spider数据集上检测F1达0.89,显著提升纠错效果。
- 适合需要高精度生成SQL的数据库交互场景。
尽管大语言模型在文本转SQL任务中表现优异,但初始生成的SQL仍常含语法和语义错误。现有纠错方法存在两大缺陷:(i)现代LLM极少产生可触发调试的显式执行错误;(ii)自校正缺乏基于问题与模式的错误建模,导致误检率高且易引入幻觉,破坏正确SQL。本文提出ErrorLLM框架,专门用于文本转SQL纠错。我们以结构化方式表示用户问题与数据库模式,采用静态检测识别执行失败与表征不匹配,并在误差空间中引入专有错误标记,捕捉分类后的隐式语义错误类型。通过精心设计的训练策略,显式建模这些错误,使模型能通过预测特定错误标记来检测复杂隐式错误。基于检测结果,引导模型对SQL结构进行误差导向修正。大量实验表明,ErrorLLM在骨干模型基础上实现最大改进。进一步分析显示,检测质量直接影响纠错效果,ErrorLLM在保持修正有效性的前提下,达到0.89的检测F1分数。
原文摘要 · Abstract (English)
Despite the remarkable performance of large language models (LLMs) in text-to-SQL (SQL generation), correctly producing SQL queries remains challenging during initial generation. The SQL refinement task is subsequently introduced to correct syntactic and semantic errors in generated SQL queries. However, existing paradigms face two major limitations: (i) self-debugging becomes increasingly ineffective as modern LLMs rarely produce explicit execution errors that can trigger debugging signals; (ii) self-correction exhibits low detection precision due to the lack of explicit error modeling grounded in the question and schema, and suffers from severe hallucination that frequently corrupts correct SQLs. In this paper, we propose ErrorLLM, a framework that explicitly models text-to-SQL Errors within a dedicated LLM for text-to-SQL refinement. Specifically, we represent the user question and database schema as structural features, employ static detection to identify execution failures and surface mismatches, and extend ErrorLLM's semantic space with dedicated error tokens that capture categorized implicit semantic error types. Through a well-designed training strategy, we explicitly model these errors with structural representations, enabling the LLM to detect complex implicit errors by predicting dedicated error tokens. Guided by the detected errors, we perform error-guided refinement on the SQL structure by prompting LLMs. Extensive experiments demonstrate that ErrorLLM achieves the most significant improvements over backbone initial generation. Further analysis reveals that detection quality directly determines refinement effectiveness, and ErrorLLM addresses both sides by high detection F1 score while maintain refinement effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。