arXiv:2604.21214cs.DBcs.AI2026-04

SQLyzr平台可细粒度评估文本转SQL模型,支持真实场景测试与错误分析。

A Demonstration of SQLyzr: A Platform for Fine-Grained Text-to-SQL Evaluation and Analysis

论文配图:A Demonstration of SQLyzr: A Platform for Fine-Grained Text-to-SQL Evaluation and Analysis
图 1 · 摘自论文原文
  • 引入多维度指标和真实数据库使用模式进行评估
  • 支持按查询类型分类、错误诊断与工作负载增强
  • 适合研究者和开发者迭代优化文本转SQL模型

随着大语言模型(LLMs)的应用,文本转SQL模型性能显著提升,已广泛用于实际场景。然而现有基准多依赖单一总分,缺乏真实场景设置,且对模型行为的分析深度不足。本文提出SQLyzr,一个全面的文本转SQL评估平台,包含多样评价指标,能捕捉生成查询的多个方面,并通过匹配真实世界SQL使用模式与数据库规模实现更贴近实际的评估。平台支持细粒度查询分类、错误分析及工作负载扩充,帮助用户深入诊断并改进模型。演示展示其交互式界面,支持自定义评估参数、查看细粒度报告及探索扩展功能。我们期望SQLyzr能解决现有基准的关键局限,推动文本转SQL模型的评估与迭代优化。代码开源:https://github.com/sepideh-abedini/SQLyzr。

原文摘要 · Abstract (English)

Text-to-SQL models have significantly improved with the adoption of Large Language Models (LLMs), leading to their increasing use in real-world applications. Although many benchmarks exist for evaluating the performance of text-to-SQL models, they often rely on a single aggregate score, lack evaluation under realistic settings, and provide limited insight into model behaviour across different query types. In this work, we present SQLyzr, a comprehensive benchmark and evaluation platform for text-to-SQL models. SQLyzr incorporates a diverse set of evaluation metrics that capture multiple aspects of generated queries, while enabling more realistic evaluation through workload alignment with real-world SQL usage patterns and database scaling. It further supports fine-grained query classification, error analysis, and workload augmentation, allowing users to better diagnose and improve text-to-SQL models. This demonstration showcases these capabilities through an interactive experience. Through SQLyzr's graphical interface, users can customize evaluation settings, analyze fine-grained reports, and explore additional features of the platform. We envision that SQLyzr facilitates the evaluation and iterative improvement of text-to-SQL models by addressing key limitations of existing benchmarks. The source code of SQLyzr is available at https://github.com/sepideh-abedini/SQLyzr.

文本转SQL模型评估数据分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。