让大模型选例更智能，不被数据库细节干扰
Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection
- 用查询结构建模语义意图，避开数据库表字段干扰
- 在Spider数据集上执行准确率提升2.9个百分点
- 适合做文本转SQL且需避免领域偏见的场景
上下文学习（ICL）中，大语言模型通过提示中的任务示例获得性能提升。但针对复杂或多模态任务，选择最优示例并非易事，因输入输出分布差异显著。本文提出一种方法，将自然语言问题与SQL查询映射到共享嵌入空间，形成任务特定表示。该方法名为MARLO（Metadata-Agnostic Representation Learning for Text-to-SQL），利用查询结构建模查询意图，避免过度依赖数据库元数据（如表名、列名或领域实体）。这使得示例选择更关注结构与语义相关性，而非特定领域或提问方式。在Spider基准测试中，基于问题相似性检索示例时，MARLO相比通用嵌入模型平均提升2.9%执行准确率；相较于次优的元数据掩码方法，准确率再提升0.8%点，且推理延迟显著更低。
原文摘要 · Abstract (English)
In-context learning (ICL) is a powerful paradigm where large language models (LLMs) benefit from task demonstrations added to the prompt. Yet, selecting optimal demonstrations is not trivial, especially for complex or multi-modal tasks where input and output distributions differ. We hypothesize that forming task-specific representations of the input is key. In this paper, we propose a method to align representations of natural language questions and those of SQL queries in a shared embedding space. Our technique, dubbed MARLO - Metadata-Agnostic Representation Learning for Text-tO-SQL - uses query structure to model querying intent without over-indexing on underlying database metadata (i.e. tables, columns, or domain-specific entities of a database referenced in the question or query). This allows MARLO to select examples that are structurally and semantically relevant for the task rather than examples that are spuriously related to a certain domain or question phrasing. When used to retrieve examples based on question similarity, MARLO shows superior performance compared to generic embedding models (on average +2.9\%pt. in execution accuracy) on the Spider benchmark. It also outperforms the next best method that masks metadata information by +0.8\%pt. in execution accuracy on average, while imposing a significantly lower inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。