小模型通过推理计算提升语法约束文本转SQL性能,但大模型仍更有效。
Beam Search, Self-Consistency, and the Limits of Inference-Time Scaling for Grammar-Constrained Text-to-SQL in Small Language Models
- 用变宽束搜索和采样投票法增加推理计算量
- 小模型上两种方法均提效,但大模型精度更高
- 束搜索比采样投票在相同算力下表现更好
本文研究小语言模型在语法约束下的文本转SQL任务中,模型规模与推理计算之间的权衡。使用0.5B至7B参数的Qwen2.5-Instruct模型(4比特量化),在Spider基准上测试束搜索(可调束宽)和采样+投票(可调采样数)两种推理增强策略。在1034个开发集样本上发现:(a) 两种方法均提升准确率,尤其对小模型显著;(b) 增加模型规模通常优于增加推理计算;(c) 在相同推理预算下,束搜索优于采样投票。该结果与无约束情况相反,表明语法约束使推理策略效果发生根本变化。
原文摘要 · Abstract (English)
One common trade-off in the use of large language models involves reducing the size of the model while increasing the amount of computation at inference time, for example by using a wider beam search. In this paper, we examine the constrained case of this "model size vs. inference compute" trade-off, in which the model outputs are constrained by a strict grammar at inference time. Our results demonstrate that the constrained trade-off behaves differently from the unconstrained trade-off. We investigate the task of converting a prose query into an equivalent SQL query (text-to-SQL). Performance is evaluated on the Spider text-to-SQL benchmark, using the Qwen2.5-Instruct model family ranging in size from 0.5B to 7B parameters, all at 4-bit precision. We experiment with two approaches to varying inference compute: (i) beam search with a variable number of beams; and (ii) sample+vote, i.e., sampling several constrained outputs and then voting on their execution results, where the number of samples is varied. On the 1034-example development set, we find that: (a) both beam search and sample+vote improve accuracy, especially on smaller model sizes; (b) the "model size vs.\ inference compute" trade-off is not advantageous in this experiment, because moving to a larger model size typically results in higher accuracy than increasing inference compute on the same model size; (c) beam search outperforms sample+vote at a matched inference budget. This latter result is of particular interest since it contrasts with the findings of the unconstrained trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。