arXiv:2607.13311cs.CL2026-07

为大规模数据库设计高效表列检索方法,提升自然语言到SQL的准确率。

Finding the Right Tables and Columns: A Benchmark and Corpus-Adaptive Embeddings for SQL Schema Retrieval

  • 基于真实企业级数据库构建表列检索基准,支持多粒度查询。
  • 提出语料自适应微调,使305M模型在10个召回率上达75.6%。
  • 无需标注数据,可适配不同数据库,适合工业部署。

SQL检索常被当作从大量SQL语句中找答案的任务,但在大规模场景下,更基础的问题是模式检索——从数千个表列中定位自然语言问题所需的表与列,这超出了模型上下文容量。本文将五个文本转SQL数据集(Spider、BIRD、BEAVER及两个LiveSQLBench变体)重构为表和列级别的检索任务,涵盖真实企业级模式与两种文档表示方式,发现现有文本与代码嵌入模型在此任务表现不佳。为此提出语料自适应微调:直接从目标语料库生成自然语言查询,构建粒度感知的困难负样本,并对305M参数嵌入模型进行对比学习微调。该方法使平均召回率@10从60.4提升至75.6(nDCG@10从51.9升至68.0),在10亿参数以下为最强检索器,性能媲美4-8B参数的先进模型。同一方法将8B模型召回率@10从77.8提升至78.4,达到当前最佳水平,表明该策略对主干网络不敏感。留一语料排除实验与泄露审计显示,性能提升源于可迁移的模式检索能力,而非数据记忆。结果确立了模式链接作为独立检索任务的地位,且轻量级、无标签的语料自适应是企业级部署的有效路径。

原文摘要 · Abstract (English)

Retrieval in the SQL setting has largely been studied as the task of finding, within a large collection of SQL statements, the statement that answers a natural-language question. At scale, however, a more fundamental retrieval problem precedes generation: schema retrieval, identifying the tables and columns a question requires in a database that may contain thousands of them, far more than fit in a model's context. We argue that this step warrants first-class evaluation. To this end, we recast five text-to-SQL datasets (Spider, BIRD, BEAVER, and two LiveSQLBench variants) as retrieval tasks at both table and column granularity, covering realistic and enterprise-scale schemas under two document representations, and we show that off-the-shelf text and code embedders transfer poorly to this setting. We then propose corpus-adaptive fine-tuning: natural-language queries are synthesized directly from the target schema corpus, granularity-aware hard negatives are mined, and a 305M-parameter embedder is fine-tuned contrastively. This procedure raises average recall@10 from 60.4 to 75.6 (nDCG@10 from 51.9 to 68.0), making the 305M model the strongest retriever under one billion parameters and competitive with state-of-the-art embedders of 4-8B parameters, more than an order of magnitude larger. The same recipe improves an 8B state-of-the-art embedder from 77.8 to 78.4 recall@10, matching the best result on the benchmark and indicating that the adaptation is backbone-agnostic. Leave-one-corpus-out experiments and a leakage audit show that these gains reflect a transferable schema-retrieval ability rather than memorization of the evaluation data. Our results establish schema linking as a standalone retrieval task and lightweight, label-free corpus adaptation as a practical route to deploying it at enterprise scale.

SQL检索模式匹配嵌入模型数据库

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。