arXiv:2606.28601cs.DBcs.AI2026-06

压缩真实数据库冗余信息,提升文本转SQL准确率

Database Context Compression for Text-to-SQL on Real-World Large Databases

论文配图:Database Context Compression for Text-to-SQL on Real-World Large Databases
图 1 · 摘自论文原文
  • 提出数据库上下文压缩方法,统一处理重复列、相似表等冗余结构
  • 在Spider 2.0-Snow上减少输入量98.7%,提升模式链接召回率达56.5%
  • 可插件式接入现有系统,适合企业级复杂数据库场景使用

Text-to-SQL在真实企业数据集(如Spider 2.0和BIRD)上的表现远低于经典学术数据集。我们指出瓶颈已从推理转向数据库表示。真实数据库包含重复审计列、大量相似表、仅文档中记录语义的模糊标识符以及大量无关数据字典。现有查询感知方法虽过滤原始上下文,但仍处理冗余且冗长的表示。本文将问题重定义为数据库上下文压缩,即一种与查询无关的转换,将模式、语义描述和外部文档重构为紧凑表示。我们提出SGCF(支持-增益组件分解)原则,统一重复列提取、同构表模板化、语义组件化与证据净化。基于此,提出DBCC——一种数据库端中间件,实现离线结构与语义压缩及轻量在线证据净化。该方法模型无关,可集成至现有Text-to-SQL流程。在Spider 2.0-Snow和BIRD上,输入上下文减少达两个数量级(最大子集从260万降至3.47万词元),在DeepSeek-V3.2下模式链接严格召回率由0%提升至56.5%(Claude Opus 4.7下达63.1%),并使三种近期Text-to-SQL系统端到端执行准确率提升1.8-1.9%。

原文摘要 · Abstract (English)

Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.0 and BIRD remains far below that on classical academic datasets. We argue that the main bottleneck is no longer reasoning, but database representation. Real databases contain repeated audit columns, large groups of similar tables, opaque identifiers whose meanings are stored only in documentation, and extensive data dictionaries with little query-relevant information. Existing query-aware methods, including schema linking and retrieval-based schema selection, filter this raw context but still operate on redundant and verbose representations. We reformulate the problem as database context compression, a query-agnostic transformation that rewrites schemas, semantic descriptions, and external documentation into a compact representation. We formalize this transformation with the SGCF (Support-Gain Component Factorization) principle, which unifies repeated column extraction, isomorphic table templating, semantic componentization, and evidence purification under a single coverage objective. Based on SGCF, we propose DBCC, a database-side middleware that performs offline structural and semantic compression together with lightweight online evidence purification. DBCC is model-agnostic and can be integrated into existing Text-to-SQL pipelines. On Spider 2.0-Snow and BIRD, DBCC reduces input context by up to two orders of magnitude (from 2.6M to 34.7K tokens on the largest Spider 2.0-Snow subset), improves schema-linking strict recall from 0% to 56.5% under DeepSeek-V3.2 (63.1% under Claude Opus 4.7), and consistently increases end-to-end execution accuracy by 1.8-1.9% over three recent Text-to-SQL systems. Our code is open-sourced at https://github.com/MrBlankness/SchemaCompression.

文本转SQL数据库压缩模式链接企业数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。