测试时扩展策略能提升Text2SQL准确率，但效果受模型和流程复杂度影响。
Rethinking Agentic Workflows: Evaluating Inference-Based Test-Time Scaling Strategies in Text2SQL Tasks
- 用分治提示和少样本示范提升模型表现
- 推理型模型在基准上准确率最高，达83.6%
- 流程越复杂，性能越不稳定，适合部署者参考
大型语言模型（LLMs）正越来越多地用于文本转SQL（Text2SQL）系统，使非专业用户能够通过自然语言查询工业数据库。尽管测试时扩展策略在基于LLM的解决方案中展现出潜力，但在真实应用场景下，尤其针对最新推理型模型，其有效性仍不明确。本文评估了六种轻量级、面向工业场景的测试时扩展策略，以及四种LLM（包括两种推理型模型），在BIRD Mini-Dev基准上的表现。除标准准确率外，还报告了推理延迟和令牌消耗，为实际系统部署提供参考。结果表明，分治提示和少样本示范对通用及推理型模型均能持续提升性能；但增加额外工作流步骤效果参差不齐，基础模型选择起决定性作用。本研究揭示了在部署Text2SQL系统时，准确性、效率与复杂性之间的实用权衡。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly powering Text-to-SQL (Text2SQL) systems, enabling non-expert users to query industrial databases using natural language. While test-time scaling strategies have shown promise in LLM-based solutions, their effectiveness in real-world applications, especially with the latest reasoning models, remains uncertain. In this work, we benchmark six lightweight, industry-oriented test-time scaling strategies and four LLMs, including two reasoning models, evaluating their performance on the BIRD Mini-Dev benchmark. Beyond standard accuracy metrics, we also report inference latency and token consumption, providing insights relevant for practical system deployment. Our findings reveal that Divide-and-Conquer prompting and few-shot demonstrations consistently enhance performance for both general-purpose and reasoning-focused LLMs. However, introducing additional workflow steps yields mixed results, and base model selection plays a critical role. This work sheds light on the practical trade-offs between accuracy, efficiency, and complexity when deploying Text2SQL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。