仅凭数据值重建数据库列语义,提升无文档仓库的可用性。
Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses
- 用语言模型结合确定性证据验证框架,从数据中推断列含义。
- 在11个BIRD库上实现42%列的0.475准确率,覆盖提升25.7个百分点。
- 适合处理无文档的生产级数据库,尤其临床数据场景。
Text-to-SQL基准数据集的列名已说明其语义,而生产环境仓库则常使用晦涩标识符且缺乏文档。本文提出罗塞塔(Rosetta)系统,通过将语言模型嵌入验证框架,仅从数据本身恢复列语义。该系统采用确定性分析器提取结构化证据(如值指纹、26种模式库、校验和判别),语言模型据此生成语义并附带出处与置信度。在11个BIRD数据库共680个列对上,去除列名后,系统在42%的列上达到0.475准确率,显著优于直接使用模型的0.223(覆盖94%)。当双方均有输出时,文本质量无差异,优势在于选择:确定性证据决定是否输出(覆盖率+0.257 [0.128, 0.378]),而非输出质量。该层为能力检测器,非增强器。更换主干模型后,文本表现可复现,但提示式放弃不转移;强制提交门控(预测先行注册,测试于第三主干及预留数据库)使无证据覆盖降至0.000。在盲测i2b2临床仓库中,系统仅凭值解码出134个真实ICD-9码中的95.5%,并对所有44个NDC药物码选择不回答。该目录支持查询时的校准弃权:全模式不可见下,朴素翻译器执行准确率从0.92降至0.42,而本系统以86%准确率实现59%覆盖率。负面结果亦如实报告,包括自身权威梯度并非核心机制。
原文摘要 · Abstract (English)
Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean. Production warehouses are the inverse: cryptic identifiers, partial or absent documentation. We address the problem they pose first: recovering what columns and values mean from the data itself. Rosetta places a language model inside a verification harness: a deterministic profiler extracts structural evidence (value fingerprints, a 26-pattern library, checksum verdicts), the model proposes semantics conditioned on that evidence, and every fact carries provenance and a confidence bounded by its evidence class. Against human documentation on 680 paired columns across eleven BIRD databases, identifiers destroyed, the harness delivers metadata that is 0.475 accurate on the 42% of columns it commits to, against 0.223 on 94% for the same model used directly. Restricted to the 283 columns where both arms speak, the harness writes no better prose than the model alone; the gain is selection: deterministic evidence governs whether the system speaks (coverage +0.257 [0.128, 0.378]), not how well. The deterministic layer is a competence detector, not a competence amplifier. A backbone swap bounds the claim: the prose finding reproduces, but prompt-requested abstention does not transfer; a code-enforced commit gate (predictions registered first; measured on a third backbone and held-out databases) makes no-evidence coverage 0.000 on every backbone. On a blind i2b2 clinical warehouse Rosetta decodes 95.5% of 134 real ICD-9 codes from values alone and abstains on all 44 NDC drug codes. The catalog supports calibrated abstention at query time: under full schema opacity a naive translator falls from 0.92 to 0.42 execution accuracy while our gate answers at 86% accuracy over 59% coverage. Negative results are reported plainly, including that our own authority ladder is not the mechanism behind the headline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。