用元数据嵌入解决企业文本转SQL的表名误选问题
Schema-First Retrieval: Embedding Catalogs for Natural Language Analytics
- 将数据库元数据（表/列/关系等）转化为向量，先选对表再生成SQL
- 在CRUSH4SQL上表召回率达96.4%@20，列召回率提升11.1点
- 适合需要精准查询的企业级数据分析系统使用
企业级文本转SQL系统常在生成SQL前就失败：模型获取了错误的模式上下文。现代数据仓库包含数千张表、缩写的字段名、非正式的指标定义、隐藏的关联规则和权限边界，这些都无法通过原始表名体现。本文提出Schema-First Retrieval，一个将数据目录元数据而非数据行进行嵌入的检索层。系统索引五类有类型对象：表、列、指标、关系和查询历史，采用对象特异性文本模板。查询时，结合并行向量搜索、谱系扩展、交叉编码器重排序、工作负载记忆和确定性访问控制门，再生成SQL。在CRUSH4SQL（1,534个问题）上，表召回率@20达到96.4%，交叉编码器重排序使列召回率@10提升11.1点；相比同等模板的BM25基线，语义检索在表召回率@5上高出32.8点。在SEDE（857个问题）上，查询历史将表召回率@5从52.1%提升至92.3%。在BIRD（96个问题）上，基于模式的上下文使SQL执行错误率从15.6%降至6.2%，降低2.5倍。结果表明，目录选择是自然语言分析中的首要检索问题，而非提示工程细节。
原文摘要 · Abstract (English)
Enterprise text-to-SQL systems often fail before SQL is generated: the model receives the wrong schema context. Modern warehouses contain thousands of tables, abbreviated columns, informal metrics, hidden join conventions, and permission boundaries that are not captured by raw table names. We introduce Schema-First Retrieval, a retrieval layer that embeds catalog metadata rather than warehouse rows. The system indexes five typed catalog objects, tables, columns, metrics, relationships, and query history, using object-specific text templates. At query time, it combines parallel vector search, lineage expansion, cross-encoder reranking, workload memory, and deterministic access-control gates before SQL generation. On CRUSH4SQL (1,534 questions), Schema-First Retrieval reaches 96.4% table recall@20 and cross-encoder reranking adds +11.1 points at column recall@10; against an equally-templated BM25 baseline, semantic retrieval is +32.8 points at table recall@5. On SEDE (857 questions), query history raises table recall@5 from 52.1% to 92.3%. On BIRD (96 questions), schema-first context reduces SQL execution errors from 15.6% to 6.2%, a 2.5x reduction. These results show that catalog selection is a first-class retrieval problem for natural language analytics, not a prompt formatting detail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。