用注意力生成数据库字段关联,速度超快且精度高
AttnLink: Turning Attention into Schema Links for Text-to-SQL

- 将大模型注意力转化为字段相关性分数,单次推理完成全部匹配
- 在Spider等数据集上达到99.22%的链接准确率,毫秒级延迟
- 支持事后调整精度与召回率,适合需要可控输出的场景
Schema linking是Text-to-SQL系统的关键组件,但现有方法常在上下文建模能力、评分可控性与推理效率之间权衡。我们提出AttnLink,一种基于注意力的框架,将大语言模型内部注意力转换为连续的相关性得分,用于匹配数据库字段。该方法从生成起始位置提取注意力至候选字段片段,实现所有候选字段在一次prefill阶段排序,无需自回归解码。我们设计了两种变体:AttnLink-U直接探测预训练注意力,无需参数更新;AttnLink-S通过监督学习对齐注意力分布与真实字段。为提升多字段覆盖能力,AttnLink-S结合集合质量目标与自适应概率下限正则化。所得得分支持通过温度缩放和累积质量选择进行事后精度-召回控制。在Spider、BIRD和Spider2-SQLite上的实验表明,AttnLink-S分别取得99.22%、95.95%和83.29%的mAP,schema-linking延迟为毫秒级。在九组生成器-数据集组合中,有七组获得最佳或并列最佳执行准确率。
原文摘要 · Abstract (English)
Schema linking is a critical component of Text-to-SQL systems, but existing approaches often trade off contextual modeling capacity, score-based controllability, and inference efficiency. We introduce AttnLink, an attention-based framework that converts LLMs' internal attention into continuous relevance scores for schema items. AttnLink extracts the attention from the generation-start position to candidate schema spans, enabling all candidates to be ranked in a single prefill pass without autoregressive decoding. We develop two variants: AttnLink-U, which directly probes pretrained attention without parameter updates, and AttnLink-S, which aligns the attention distribution with gold schema items through direct supervision. To improve coverage of multiple relevant schema items, AttnLink-S combines a set-mass objective with an adaptive probability-floor regularizer. The resulting scores support post-hoc precision-recall control through temperature scaling and cumulative-mass selection. Experiments on Spider, BIRD, and Spider2-SQLite show that AttnLink-S achieves mAP scores of 99.22%, 95.95%, and 83.29%, respectively, with millisecond-scale schema-linking latency. It also yields the best or tied-best execution accuracy for downstream SQL generation in seven of nine generator-dataset settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。