用大模型生成高保真测试数据,解决复杂SQL生成服务的测试难题。
High-Fidelity And Complex Test Data Generation For Google SQL Code Generation Services
- 基于大模型+预处理后处理,生成符合复杂结构约束的数据
- 支持包含嵌套字段、多表连接等复杂查询的完整测试覆盖
- 适合工业级SQL生成服务,尤其在无法获取生产数据时
在工业场景中,由于生产数据难以获取,对高保真测试数据的需求尤为迫切。传统方法在生成低保真数据方面表现不佳,难以建模复杂的结构和语义关系,这对自然语言转SQL(NL2SQL)等复杂SQL生成服务的测试构成挑战。本文针对谷歌工作负载中常见的嵌套列等复杂数据结构,提出一种基于大语言模型(LLM)的测试数据生成方法,结合策略性预处理与后处理步骤,生成语法正确且语义相关的高保真数据。该方法可有效支持包含多表连接、聚合及深层嵌套子查询的复杂SQL查询测试,保障如NL2SQL和SQL Code Assistant等服务的全面评估。实验表明,基于大模型(Gemini)的测试数据生成方案在缺乏生产数据的场景下具备显著实用性。
原文摘要 · Abstract (English)
The demand for high-fidelity test data is paramount in industrial settings where access to production data is largely restricted. Traditional data generation methods often fall short, struggling with low-fidelity and the ability to model complex data structures and semantic relationships that are critical for testing complex SQL code generation services like Natural Language to SQL (NL2SQL). In this paper, we address the critical need for generating syntactically correct and semantically relevant high-fidelity mock data for complex data structures that includes columns with nested structures that we frequently encounter in Google workloads. We highlight the limitations of existing approaches used in production, particularly their inability to handle large and complex data structures, as well as the lack of semantically coherent test data that lead to limited test coverage. We demonstrate that by leveraging Large Language Models (LLMs) and incorporating strategic pre- and post-processing steps, we can generate syntactically correct and semantically relevant high-fidelity test data that adheres to complex structural constraints and maintains semantic integrity to the SQL test targets (queries/functions). This approach supports comprehensive testing of complex SQL queries involving joins, aggregations, and even deeply nested subqueries, ensuring robust evaluation of SQL code generation services, like NL2SQL and SQL Code Assistant. Our results demonstrate the practical utility of an LLM (\textit{Gemini}) based test data generation for industrial SQL code generation services where generating high-fidelity test data is essential due to the frequent unavailability and inaccessibility of production datasets for testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。