对比推理与非推理大模型在Text-to-SQL中的实际云成本，发现速度快不等于省钱。
Cost Trade-offs of Reasoning and Non-Reasoning Large Language Models in Text-to-SQL
- 在StackOverflow数据集上测试6个LLM，评估真实云查询成本
- 推理模型处理数据量少44.5%，准确率仍达96.7%~100%
- 非推理模型成本波动超3.4倍，最大单次消耗36GB，需优化部署
尽管Text-to-SQL系统已实现高准确率，但现有效率指标如有效效率得分过度关注执行时间，而我们发现该指标与基于用量的云计费机制根本脱节。本文通过在Google BigQuery上对六种大语言模型执行180次Text-to-SQL查询，使用230 GB的StackOverflow数据集，评估了推理与非推理模型间的云查询成本权衡。分析显示，推理模型比非推理模型少处理44.5%的数据字节，同时保持96.7%至100%的正确率；执行时间与查询成本相关性极弱（r=0.16），表明速度优化未必带来成本节约。非推理模型成本波动极大，最高达3.4倍，单次查询最高消耗36GB，超过最优模型平均值1.8GB的20倍以上，主要因缺失分区过滤和低效连接。本文识别出这些常见低效模式，并提供企业级成本敏感场景下的部署建议。
原文摘要 · Abstract (English)
While Text-to-SQL systems achieve high accuracy, existing efficiency metrics like the Valid Efficiency Score prioritize execution time, a metric we show is fundamentally decoupled from consumption-based cloud billing. This paper evaluates cloud query execution cost trade-offs between reasoning and non-reasoning Large Language Models by performing 180 Text-to-SQL query executions across six LLMs on Google BigQuery using the 230 GB StackOverflow dataset. Our analysis reveals that reasoning models process 44.5% fewer bytes than non-reasoning counterparts while maintaining equivalent correctness at 96.7% to 100%, and that execution time correlates weakly with query cost at $r=0.16$, indicating that speed optimization does not imply cost efficiency. Non-reasoning models also exhibit extreme cost variance of up to 3.4$\times$, producing outliers exceeding 36 GB per query, over 20$\times$ the best model's 1.8 GB average, due to missing partition filters and inefficient joins. We identify these prevalent inefficiency patterns and provide deployment guidelines to mitigate financial risks in cost-sensitive enterprise environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。