让大模型自动判断何时停止生成SQL,避免无效重复。
Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQL

- 用轻量一维模型跟踪一致性变化,动态决定是否继续运行。
- 在多个数据集上提前停止,平均节省40%以上调用次数。
- 支持乱序训练增强,对错误判断也保持稳定预测能力。
重复调用大模型是评估文本转SQL结果可信度的常用方法:多次运行流水线,检查每条SQL执行结果,以判断结论的一致性作为置信度信号。核心问题在于何时停止——何时一致性已收敛?本文将此建模为收敛预测问题,训练一系列轻量级一维模型,实时观察一致性轨迹,在每一步判断后续运行是否可能显著改变结果,并与基于贝塔-伯努利分布的理论停止规则及学习型运行次数基线进行对比。在BIRD基准和两个真实客户数据集上,该方法能根据用户问题自适应调整停止点:一致性早收敛时提前终止,晚收敛时持续运行。进一步发现,各次运行间存在弱序列相关性,可通过对运行顺序打乱作为训练增强,由可调参数控制打乱程度。方法在三个数据集上表现稳定;即使在生成结果与真实答案对比时引入噪声模拟不完美判别器,仍能可靠预测收敛状态。
原文摘要 · Abstract (English)
Repeated LLM calls are the standard way to estimate how trustworthy a Text-to-SQL result is: run the pipeline multiple times, judge each SQL execution, and use the consistency of the verdicts as a confidence signal. The open question is when to stop, when the consistency has converged. We formulate this as a convergence-prediction problem and train a family of lightweight 1-D models that observe the running consistency trajectory and decide, at each step, whether further runs are unlikely to shift it materially, and we benchmark them against a principled Beta-Bernoulli stopping rule and a learned run-count baseline. On the BIRD benchmark and two production customer datasets, our method adapts its stopping point to each user question, halting sooner when consistency converges early and continuing longer when it converges late. We further show that the weak serial correlation between runs lets us permute their order as a training augmentation, controlled by a tunable shuffling weight. Performance stays consistent across the three datasets, and to mimic an imperfect production judge we inject noise into the correct/incorrect verdicts obtained by comparing the generated and ground-truth SQL results, showing that the method still predicts convergence reliably.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。