根据任务难度动态调整探索范围，提升大模型文本转SQL的准确率。
CA-SQL: Complexity-Aware Inference Time Reasoning for Text-to-SQL via Exploration and Compute Budget Allocation
- 按任务复杂度动态分配计算资源，扩大解空间探索
- 在BIRD挑战集上达51.72%准确率，仅用GPT-4o-mini
- 适合需要高精度推理的复杂SQL生成场景
尽管近期推理时学习进展提升了大模型在文本转SQL任务中的表现，当前方法在Bird-Bench（BIRD）基准中最难的任务上仍表现不佳。这源于解空间探索不足，难以发现可进一步优化的候选查询。为此，我们提出CA-SQL，一种新型文本转SQL流水线，通过估计任务难度动态调整生成候选解的探索广度。此外，我们采用基于进化搜索原理的定制提示种子方法，激发基础大模型的探索行为，并设计新颖的投票机制，在搜索结束时选出最优候选解。实验表明，该方法在BIRD开发集“挑战”层级问题上取得51.72%的最新成绩，仅使用GPT-4o-mini，优于其他上下文学习方法，即使后者使用更大模型。整体上，方法在BIRD开发集达到61.06%执行准确率和68.77% Soft F1得分。
原文摘要 · Abstract (English)
While recent advancements in inference-time learning have improved LLM reasoning on Text-to-SQL tasks, current solutions still struggle to perform well on the most challenging tasks in the Bird-Bench (BIRD) benchmark. This is due to inadequate solution space exploration, which is necessary to uncover promising candidate queries that can be further refined to produce the correct output. To address this challenge, we introduce CA-SQL, a novel Text-to-SQL pipeline that utilizes the estimated difficulty of a task to dynamically scale the breadth of the exploration for generating solution candidates. In addition, we use a custom prompt seeding method, based on principles of evolutionary search, to further elicit exploratory behavior from the base LLM and a novel voting method to select the best candidate solution at the end of the search. Experiments demonstrate that our solution achieves a state-of-the-art score of 51.72% on the "challenging" tier of BIRD development set problems, using only GPT-4o-mini, out-performing other in-context learning approaches, even those that leverage larger models. Overall, our method attains a competitive 61.06% execution accuracy and 68.77% Soft F1 score on the BIRD development dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。