arXiv:2603.21530cs.SEcs.AI2026-03中稿 · ICSE 2026 Industry…

用蒙特卡洛树搜索提升轻量级大模型生成数据库测试用例的覆盖深度和多样性。

LLM-Based Test Case Generation in DBMS through Monte Carlo Tree Search

  • 基于特征引导与错误反馈构建分层查询生成树,提升语法正确性与语义多样性。
  • 通过树搜索优化种子选择与变异规则,使代码覆盖率提升超40%。
  • 适合在隐私敏感场景下使用轻量模型进行数据库系统自动化测试。

数据库管理系统(DBMS)是现代数据驱动应用的核心基础设施,高质量的SQL测试用例对保障系统可靠性至关重要。传统方法如模糊测试虽有效,但适配不同专有SQL方言需大量手动工作。大语言模型(LLMs)为自动化生成提供了可能,但在工业环境中面临两大挑战:一是轻量级模型受限于安全与隐私,难以生成符合专有方言的合法查询;二是生成的查询语义相似,仅覆盖浅层执行路径,迅速达到覆盖瓶颈。为此,我们提出MIST框架,基于蒙特卡洛树搜索实现面向DBMS的测试用例生成。该框架包含两阶段:特征引导的错误驱动测试用例合成,通过构建层次化特征树并利用错误反馈指导生成,以产出语法正确且语义多样化的查询;以及基于蒙特卡洛树搜索的测试用例变异,联合优化种子查询选择与变异规则,依托覆盖率反馈探索更深层执行路径。在三个主流DBMS上使用四种轻量级模型的实验表明,相较于基线方法,MIST平均提升43.3%的行覆盖率、32.3%的功能覆盖率和46.4%的分支覆盖率,其中优化器模块最高行覆盖率达69.3%。

原文摘要 · Abstract (English)

Database Management Systems (DBMSs) are fundamental infrastructure for modern data-driven applications, where thorough testing with high-quality SQL test cases is essential for ensuring system reliability. Traditional approaches such as fuzzing can be effective for specific DBMSs, but adapting them to different proprietary dialects requires substantial manual effort. Large Language Models (LLMs) present promising opportunities for automated SQL test generation, but face critical challenges in industrial environments. First, lightweight models are widely used in organizations due to security and privacy constraints, but they struggle to generate syntactically valid queries for proprietary SQL dialects. Second, LLM-generated queries are often semantically similar and exercise only shallow execution paths, thereby quickly reaching a coverage plateau. To address these challenges, we propose MIST, an LLM-based test case generatIon framework for DBMS through Monte Carlo Tree search. MIST consists of two stages: Feature-Guided Error-Driven Test Case Synthetization, which constructs a hierarchical feature tree and uses error feedback to guide LLM generation, aiming to produce syntactically valid and semantically diverse queries for different DBMS dialects, and Monte Carlo Tree Search-Based Test Case Mutation, which jointly optimizes seed query selection and mutation rule application guided by coverage feedback, aiming at boosting code coverage by exploring deeper execution paths. Experiments on three widely-used DBMSs with four lightweight LLMs show that MIST achieves average improvements of 43.3% in line coverage, 32.3% in function coverage, and 46.4% in branch coverage compared to the baseline approach with the highest line coverage of 69.3% in the Optimizer module.

数据库测试大模型生成蒙特卡洛树搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。