构建首个跨方言的文本转SQL评测基准，验证模型泛化能力
UniQL: Towards Dialect-Universal Benchmarking for Text-to-SQL
- 通过迁移+翻译+验证流程构建16方言对齐数据集
- 覆盖1534个自然语言问题，生成24544条方言专属查询
- 揭示现有模型严重依赖SQLite，跨方言迁移能力差
现有文本转SQL评测基准主要基于SQLite，难以评估模型在异构SQL方言间的泛化能力。现实数据库系统在语法、函数、类型系统和执行语义上差异显著，相同自然语言意图需对应不同方言的SQL实现。本文提出UniQL，一个经人工验证的跨方言文本转SQL评测基准。该基准将1534个自然语言问题与16种SQL方言的可执行SQL标注对齐，共生成24544条方言特定查询。所有方言共享相同意图、对齐的模式和数据库内容，实现对方言泛化能力的受控评估。UniQL通过数据库迁移、SQL转换、执行引导验证、迭代规则归纳与人工校验的混合流水线构建。在开源与闭源大模型上的实验表明，当前模型距离方言通用仍有很大差距，性能在不同数据库系统间波动明显，且从SQLite取得的成功难以迁移到其他方言。这些发现凸显了建立对齐的跨方言评测基准和更具备方言感知能力的文本转SQL方法的必要性。代码与数据见https://github.com/JerryGao818/UniQL。
原文摘要 · Abstract (English)
Existing text-to-SQL benchmarks are largely centered on SQLite, making it difficult to evaluate whether models can generalize across heterogeneous SQL dialects. However, real-world database systems differ substantially in syntax, functions, type systems, and execution semantics, so the same natural language intent often requires dialect-specific SQL realizations. We introduce UniQL, a human-verified benchmark for cross-dialect text-to-SQL evaluation. UniQL aligns 1,534 natural language questions with executable SQL annotations across 16 SQL dialects, yielding 24,544 dialect-specific queries. All dialects share the same intents, aligned schemas and database contents, enabling controlled evaluation of dialect generalization. UniQL is constructed through a hybrid pipeline combining database migration, SQL translation, execution-guided verification, iterative rule summarization, and human validation. Experiments on both open-source and closed-source LLMs show that current models remain far from dialect-universal, with substantial performance variation across database systems and limited transfer from SQLite success to other dialects. These findings highlight the need for aligned cross-dialect benchmarks and more dialect-aware text-to-SQL methods. Code and data are available at https://github.com/JerryGao818/UniQL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。