arXiv:2608.17795cs.CL2026-08

提出可追溯的文本转SQL验证方法,让模型决策透明可信。

TraceSQL: Traceable Answerability Estimation for Reference-Free Text-to-SQL Verification

论文配图:TraceSQL: Traceable Answerability Estimation for Reference-Free Text-to-SQL Verification
图 1 · 摘自论文原文
  • 基于67个诊断特征构建轻量级可解释验证模型
  • 在BIRD数据集上达66.47% F1和64.48% AUC,优于基线
  • 支持逐项分析预测依据,适合需可解释性的部署场景

文本转SQL系统通常依赖真实SQL或执行结果进行评估,但在实际部署中这些监督信号不可用,导致难以验证生成的SQL是否正确。现有方法使用大模型作为判官或专用代理,但决策过程缺乏可追溯性。尽管结果奖励模型(ORM)通过学习带执行标签的候选SQL并为未见查询打分缓解了此问题,仍无法提供预测背后的信号细节。为此,我们提出TraceSQL,一种基于显式诊断特征的轻量且可追溯的验证模型。该模型融合67个特征,涵盖问题模糊性、需求、问题-模式-SQL一致性、SQL结构及意图对齐等维度,使每项预测均可追溯至具体证据。在BIRD开发数据库上,TraceSQL取得66.47% F1和64.48% ROC-AUC,优于GradeSQL-7B ORM基线(61.87% F1,58.26% AUC)。特征归因分析表明,模型同时依赖语义基础与确定性SQL结构信号。结果证明,轻量级学习模型可在保持可解释性的同时实现有效验证。

原文摘要 · Abstract (English)

Text-to-SQL systems are commonly evaluated using ground-truth SQL queries or reference execution results, but such supervision is unavailable at inference time in real-world deployments. This creates a critical verification problem: given only a user question, database context, and generated SQL, can a system estimate whether the generated query is likely to correctly answer the question? Recent approaches use LLMs as judge or specialized agents to inspect generated SQL, but their decisions can be difficult to trace. Outcome Reward Models (ORMs) address this by learning from execution-labeled candidate SQLs and assigning correctness scores to unseen queries, yet they still provide limited visibility into the signals behind each verification. To address this limitation, we propose TraceSQL, a lightweight and traceable verification model built on explicit diagnostic features. TraceSQL combines 67 features capturing question ambiguity, question requirements, question-schema-SQL consistency, SQL structure, and intent alignment. These signals remain available for examining which factors influence each prediction and for tracing decisions back to diagnostic evidence. On BIRD development databases, TraceSQL achieves 66.47% F1 and 64.48% ROC-AUC, compared with 61.87% F1 and 58.26% ROC-AUC for the GradeSQL-7B ORM baseline on the same generated-SQL evaluation. Feature attribution further shows that the model relies on both semantic grounding and deterministic SQL-structure signals. These results show that SQL verification can be performed with a lightweight learned model while retaining feature-level evidence for inspecting and diagnosing its predictions.

文本转SQL可解释性验证模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。