用历史查询数据生成可复用的SQL参考卡片,显著提升企业级文本转SQL效果。
Beyond the Harness: End-to-End Optimization of Context Artifacts for Enterprise Text-to-SQL

- 从历史查询中自动构建动态知识上下文,替代固定提示模板。
- 在真实电商数据上使查询相似度提升12%至25%,优于优化检索组件。
- 适合需要高精度文本转SQL的企业系统,尤其有大量历史查询数据的场景。
将大语言模型用于企业级文本转SQL的瓶颈不在于模型本身,而在于输入给模型的上下文信息:业务逻辑涉及数千张表,模型无法一次性处理完整数据库目录。本文认为最有效的干预点是模型所消费的知识库上下文,且该上下文应基于历史使用数据动态构建,而非固定设定。我们采用查询图(query-DAG)分解方法——与BEAVER等企业基准标注的中间结构同源,但此处从生产环境的SQL中恢复——对比了理想查询图与检索到的知识库上下文的价值。结果表明,在完整理想图基础上,加入检索到的知识库上下文带来的增益最大。基于此,我们设计了一种蒸馏流程,将历史查询特征转化为可复用的SQL参考卡片。在某大型在线零售商的5176条生产查询基准上,优化这些上下文工件带来的提升(约12%–25%的AST相似度)远超优化检索组件(约3%–12%)。在公开的BEAVER基准上,仅使用表卡片的效果与原始历史SQL相当;最优方案同时检索卡片和原始SQL,得分9.00%对基准6.33%(p值0.12),在未引入代理循环的情况下实现显著改进。
原文摘要 · Abstract (English)
Deploying LLMs for enterprise Text-to-SQL is bottlenecked less by the model than by what context reaches it: business logic spans thousands of tables, and no model can ingest a full catalog at once. We argue that the most effective place to intervene is therefore the \emph{knowledge-base context} the model consumes, and that this context should be \emph{constructed} from historical usage rather than tuned for as a fixed input. Using a query-DAG decomposition--the same family of intermediates that enterprise benchmarks like BEAVER annotate, here recovered from production SQL--we compare the value of oracle query graphs versus retrieved knowledge-base context. In this ablation, retrieved knowledge-base context provides the largest marginal improvement when added to the full oracle graph. Building on this, we optimize a distillation procedure that turns historical query profiles into reusable SQL reference cards. On a benchmark of 5176 production queries from a major online retailer, optimizing these context artifacts yields larger gains (${\sim}12$--$25\%$ AST similarity) than optimizing the retrieval harness (${\sim}3$--$12\%$). On the public BEAVER benchmark, which lacks the production-usage signals available in our internal setting, the picture is more mixed: table cards alone perform about the same as raw historical SQL. The best optimized variant retrieves both cards and raw SQL, scoring $9.00\%$ versus $6.33\%$ (p-value $0.12$) for the comparable baseline on a held-out $N{=}300$ subset, using retrieved context and harness changes but no agentic loop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。