用大模型生成真实且可定制的SQL测试集，提升数据库研究效率
SQLBarber: A System Leveraging Large Language Models to Generate Customized and Realistic SQL Workloads
- 通过自然语言描述自动生成可定制的SQL模板，无需手动编写
- 能高效生成大量符合指定执行成本分布的查询，支持任意规模
- 基于真实云数据仓库统计构建，适合数据库系统研发与优化
数据库研究与开发常需大量SQL查询用于基准测试。但因隐私问题难以获取真实SQL，现有生成方法在定制性和满足真实约束方面受限。为此，我们提出SQLBarber，一个基于大语言模型（LLMs）的系统，用于生成定制化且真实的SQL工作负载。SQLBarber (i) 无需用户预先编写SQL模板，仅需自然语言描述即可约束模板；(ii) 能高效生成大量匹配用户定义的成本分布（如基数和执行计划开销）的查询；(iii) 利用Amazon Redshift和Snowflake的执行统计数据，推导出反映真实查询特征的模板规范与成本分布。系统包含：(i) 声明式接口，轻松生成定制模板；(ii) LLM驱动的流水线，集成自校正模块，根据查询开销进行模板分析、优化与剪枝；(iii) 贝叶斯优化器，高效探索谓词取值，找到满足目标成本分布的查询集合。我们基于雪佛兰和亚马逊红移的真实统计数据，构建并开源了10个不同难度级别和目标成本分布的基准测试集。大量实验表明，SQLBarber是唯一能生成定制化模板的系统，查询生成时间降低一到三个数量级，且与目标成本分布的对齐度显著优于现有方法。
原文摘要 · Abstract (English)
Database research and development often require a large number of SQL queries for benchmarking purposes. However, acquiring real-world SQL queries is challenging due to privacy concerns, and existing SQL generation methods are limited in customization and in satisfying realistic constraints. To address this issue, we present SQLBarber, a system based on Large Language Models (LLMs) to generate customized and realistic SQL workloads. SQLBarber (i) eliminates the need for users to manually craft SQL templates in advance, while providing the flexibility to accept natural language specifications to constrain SQL templates, (ii) scales efficiently to generate large volumes of queries matching any user-defined cost distribution (e.g., cardinality and execution plan cost), and (iii) uses execution statistics from Amazon Redshift and Snowflake to derive SQL template specifications and query cost distributions that reflect real-world query characteristics. SQLBarber introduces (i) a declarative interface for users to effortlessly generate customized SQL templates, (ii) an LLM-powered pipeline augmented with a self-correction module that profiles, refines, and prunes SQL templates based on query costs, and (iii) a Bayesian Optimizer to efficiently explore different predicate values and identify a set of queries that satisfy the target cost distribution. We construct and open-source ten benchmarks of varying difficulty levels and target query cost distributions based on real-world statistics from Snowflake and Amazon Redshift. Extensive experiments on these benchmarks show that SQLBarber is the only system that can generate customized SQL templates. It reduces query generation time by one to three orders of magnitude, and significantly improves alignment with the target cost distribution, compared with existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。