arXiv:2607.06229cs.CLcs.AI2026-07

新基准测试让大模型学会在SQL中调用AI功能,提升数据分析效率。

Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows

论文配图:Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows
图 1 · 摘自论文原文
  • 构建AI原生SQL任务的自动化生成流水线,实现自然语言到带AI函数的SQL转换。
  • 顶尖闭源模型准确率达67%-70%,开源模型最高58.1%,主要错在参数设置和语义对齐。
  • 简单代理框架反而更有效,传统SQL策略不适用于AI增强型查询生成。

主流云数据平台现已将大语言模型能力作为原生SQL函数集成,使分析师可在普通SQL查询中完成分类、筛选、情感分析、信息提取、相似性搜索和聚合等操作。然而,现有text-to-SQL评测仅覆盖传统SQL,缺乏对模型生成此类AI原生SQL能力的评估。我们提出Spider 2.0-AIFunc,一个包含465个经验证实例的基准,涵盖125个真实数据库,覆盖Snowflake平台上六类AI函数。基于现有企业级text-to-SQL基准,通过基于代理的流水线,将原始任务重写为AI原生形式,同步转换目标查询并优化自然语言指令,使预期解法清晰且减少歧义。所有实例均通过跨时间窗口的多轮重复执行协议验证结果稳定性后发布。对十种前沿语言模型的评估显示,最强闭源模型执行准确率为67-70%,最佳开源模型达58.1%,差距主要源于谓词指定、模式对齐和AI函数参数化错误。专为传统text-to-SQL设计的代理框架在该任务上迁移效果不佳:最简代理配置始终表现相当或更优,表明其采用的策略在此场景下并非关键。数据已公开于https://github.com/Leolty/Spider2-AIFunc。

原文摘要 · Abstract (English)

Major cloud data platforms now expose large language model capabilities as native SQL functions, enabling analysts to perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation within ordinary SQL queries. Yet existing text-to-SQL benchmarks evaluate only conventional SQL and provide no signal on whether models can generate such AI-native SQL. We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform. Starting from an existing enterprise text-to-SQL benchmark, we construct Spider 2.0-AIFunc through an agent-based pipeline that rewrites source tasks into AI-native form, simultaneously transforming target queries and refining natural language instructions to make the intended AI-native solution explicit and reduce ambiguity. All instances pass a multi-round repeated execution protocol across temporally separated windows to confirm result stability before release. Evaluating ten state-of-the-art language models, we find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization. Agent frameworks designed for traditional text-to-SQL challenges, such as schema retrieval and relevant table selection, do not transfer effectively to AI-native SQL: a minimal agent setup consistently matches or outperforms more elaborate alternatives, suggesting that the strategies these frameworks employ are less critical in this setting. Data are available at https://github.com/Leolty/Spider2-AIFunc .

文本转SQLAI函数数据智能大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。