用图模型精简复杂数据库结构,让大语言模型更高效地生成SQL。
Scaling Text2SQL via LLM-efficient Schema Filtering with Functional Dependency Graph Rerankers
- 基于功能依赖图重排列,保留列间关联性筛选关键字段。
- 在超大规模数据库(2.3万+列)上实现近满分召回率与低延迟。
- 适合处理真实世界复杂数据库的文本转SQL任务。
现有文本转SQL系统通常将整个数据库模式(含大量列信息)作为提示输入大语言模型,但在真实场景中因模式过大超出上下文长度而失效。面对包含数百张表、数万列的Spider 2.0基准数据集,现有方法常崩溃。当前解决方案要么依赖高成本多步提示流程,要么独立排序列忽略列间结构。为此,我们提出 oolname,一个开源、高效的模式过滤框架:首先使用查询感知的LLM编码器结合值和元数据对列进行初步排序;其次通过轻量级图变压器对存在功能依赖的列进行重排序;最后采用斯坦纳树启发式算法选择保持连通性的子模式。实验表明, oolname在真实数据集上达到接近完美的召回率,精度优于CodeS、SchemaExP、Qwen重排器及嵌入检索方法,且中位延迟低于1秒,可扩展至23,000+列的数据库。代码已公开于https://github.com/thanhdath/grast-sql。
原文摘要 · Abstract (English)
Most modern Text2SQL systems prompt large language models (LLMs) with entire schemas -- mostly column information -- alongside the user's question. While effective on small databases, this approach fails on real-world schemas that exceed LLM context limits, even for commercial models. The recent Spider 2.0 benchmark exemplifies this with hundreds of tables and tens of thousands of columns, where existing systems often break. Current mitigations either rely on costly multi-step prompting pipelines or filter columns by ranking them against user's question independently, ignoring inter-column structure. To scale existing systems, we introduce \toolname, an open-source, LLM-efficient schema filtering framework that compacts Text2SQL prompts by (i) ranking columns with a query-aware LLM encoder enriched with values and metadata, (ii) reranking inter-connected columns via a lightweight graph transformer over functional dependencies, and (iii) selecting a connectivity-preserving sub-schema with a Steiner-tree heuristic. Experiments on real datasets show that \toolname achieves near-perfect recall and higher precision than CodeS, SchemaExP, Qwen rerankers, and embedding retrievers, while maintaining sub-second median latency and scaling to schemas with 23,000+ columns. Our source code is available at https://github.com/thanhdath/grast-sql.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。