arXiv:2608.28432cs.CLcs.AI2026-08

剖析文本转SQL的上下文学习模块成本效益,发现只有执行反馈优化始终高效。

Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL

论文配图:Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL
图 1 · 摘自论文原文
  • 在统一框架下测试17种模块组合,量化各环节对准确率的边际贡献
  • 执行反馈优化是唯一普遍有效且成本低的模块,其他模块依赖模型能力
  • 预算有限时,优化中等模型的复杂管道比升级顶级模型更划算

近期基于上下文学习(ICL)的文本转SQL研究通过构建日益复杂的流水线显著提升了公开基准上的执行准确率,但现有研究通常仅报告整体端到端准确率,未量化各设计选择的边际准确率-成本贡献。为此,我们在单一可控实现下,针对ICL文本转SQL流水线中的五个常见模块,实例化了17种范式级配置,并评估其在四种不同能力水平和推理风格的骨干模型上的边际贡献与成本。分析显示,执行反馈优化是唯一在所有条件下均保持高效益且成本稳定的范式;而大多数其他模块仅在特定骨干模型上有效。词元统计表明,输入需求更受流水线结构影响,输出需求则更敏感于骨干生成行为。跨模块分析进一步发现,堆叠策略在多数骨干上提升准确率,但增益构成随骨干能力差异而异。此外,我们发现,在固定预算下,为中等性能骨干设计更复杂的流水线,往往比使用前沿模型搭配简单流水线更具性价比。这些发现归纳为可操作的成本感知分层指南,可迁移至另外五种骨干模型,无需逐范式搜索。

原文摘要 · Abstract (English)

Recent advances in in-context learning (ICL) text-to-SQL have substantially improved execution accuracy on public benchmarks by assembling increasingly elaborate pipelines around the base generator, yet existing studies typically report aggregate end-to-end accuracy, without quantifying the marginal accuracy-cost contribution of individual design choices. Consequently, providing a unified, paradigm-level cost-accuracy quantification remains a critical challenge for understanding and configuring modern text-to-SQL. To address this, we instantiate 17 paradigm-level configurations across five recurring modules of the ICL text-to-SQL pipeline under a single controlled implementation, and attribute each paradigm's marginal contribution and incurred cost across all four backbones spanning diverse capability levels and reasoning styles. Our analysis reveals that execution-feedback refinement is the only paradigm whose benefit holds universally at consistently low cost, while most other modules help only under backbone-dependent conditions. Token accounting shows that input demand is more closely tied to pipeline structure, whereas output demand is more sensitive to backbone generation behavior. Cross-module analysis further shows that stacking improves accuracy on most backbones, although how the gains compose varies with backbone capability. We also find that a fixed budget is often better spent engineering a more elaborate pipeline over a mid-tier backbone than upgrading to a frontier model with a lean pipeline. These findings distill into an actionable, cost-aware tiered guideline that transfers to five additional backbones without per-paradigm search.

文本转SQL上下文学习成本分析模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。