arXiv:2505.20591cs.CLcs.DB2025-05被引 1

优化提示词中的示例选择,提升NL2SQL系统在生产环境的精度与速度。

Effectiveness of Prompt Optimization in NL2SQL Systems

  • 基于多目标优化动态筛选最具代表性的示例,替代传统相似性匹配
  • 实测表明该方法在保持高精度的同时显著降低查询延迟
  • 适合对响应速度和准确率要求高的工业级数据库查询系统

NL2SQL方法因大语言模型(LLMs)的强大能力而获益良多。构建特定领域的NL2SQL系统,只需向LLM提供足够的上下文信息,如模式详情和翻译示例。然而,要实现高准确性,仍需精心选择每条查询的上下文——包括识别相关的模式元素、单元格值及有助于理解领域特性的合适示例。基于检索的方法已成为识别此类上下文的主流方式,但其引入了额外的推理开销。本文认为,在生产场景中,系统需要的是高精度与高性能,而非仅高质量的SQL生成,这正是当前多数研究关注的重点。在此背景下,针对特定生产环境精心挑选一组静态示例——涵盖查询日志特征、目标数据库、SQL构造及执行延迟——比单纯依赖相似性进行示例选择更为关键。核心挑战在于为特定生产设置识别出具有代表性的示例集。为此,我们提出一种提示词优化框架,不仅满足高精度需求,还通过多目标优化提升生成SQL的性能。初步的实证分析证明了该框架的有效性。

原文摘要 · Abstract (English)

NL2SQL approaches have greatly benefited from the impressive capabilities of large language models (LLMs). In particular, bootstrapping an NL2SQL system for a specific domain can be as simple as instructing an LLM with sufficient contextual information, such as schema details and translation demonstrations. However, building an accurate system still requires the rigorous task of selecting the right context for each query-including identifying relevant schema elements, cell values, and suitable exemplars that help the LLM understand domain-specific nuances. Retrieval-based methods have become the go-to approach for identifying such context. While effective, these methods introduce additional inference-time costs due to the retrieval process. In this paper, we argue that production scenarios demand high-precision, high-performance NL2SQL systems, rather than simply high-quality SQL generation, which is the focus of most current NL2SQL approaches. In such scenarios, the careful selection of a static set of exemplars-capturing the intricacies of the query log, target database, SQL constructs, and execution latencies-plays a more crucial role than exemplar selection based solely on similarity. The key challenge, however, lies in identifying a representative set of exemplars for a given production setting. To this end, we propose a prompt optimization framework that not only addresses the high-precision requirement but also optimizes the performance of the generated SQL through multi-objective optimization. Preliminary empirical analysis demonstrates the effectiveness of the proposed framework.

NL2SQL提示优化生产系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。