构建表格检索基准TARGET，评估生成任务中表格召回效果。
TARGET: Benchmarking Table Retrieval for Generative Tasks
- 提出TARGET基准，专门评测生成任务中的表格检索性能。
- 密集向量检索器显著优于BM25，尤其在结构化数据上表现更优。
- 揭示元数据缺失对检索影响大，适合研究数据检索与生成的学者。
结构化数据蕴含丰富信息，对数据分析和机器学习应用至关重要。近年来，表示学习与生成模型的发展推动了自然语言到结构化数据的交互接口（如text-to-SQL）的兴起。通过检索增强生成，结合对话或智能体组件可提升回答的时效性、准确性和全面性。核心问题是：如何为分析查询精准召回相关表格？为此，我们提出TARGET：面向生成任务的表格检索基准。利用TARGET，我们独立评估不同检索器的表现及其对下游任务的影响。结果表明，基于密集嵌入的检索器远超BM25基线，后者在结构化数据上的表现不如在非结构化文本中有效。同时，我们发现检索器对元数据（如缺失表名）敏感，并揭示不同数据集与任务间存在显著性能差异。TARGET已开源：https://target-benchmark.github.io。
原文摘要 · Abstract (English)
The data landscape is rich with structured data, often of high value to organizations, driving important applications in data analysis and machine learning. Recent progress in representation learning and generative models for such data has led to the development of natural language interfaces to structured data, including those leveraging text-to-SQL. Contextualizing interactions, either through conversational interfaces or agentic components, in structured data through retrieval-augmented generation can provide substantial benefits in the form of freshness, accuracy, and comprehensiveness of answers. The key question is: how do we retrieve the right table(s) for the analytical query or task at hand? To this end, we introduce TARGET: a benchmark for evaluating TAble Retrieval for GEnerative Tasks. With TARGET we analyze the retrieval performance of different retrievers in isolation, as well as their impact on downstream tasks. We find that dense embedding-based retrievers far outperform a BM25 baseline which is less effective than it is for retrieval over unstructured text. We also surface the sensitivity of retrievers across various metadata (e.g., missing table titles), and demonstrate a stark variation of retrieval performance across datasets and tasks. TARGET is available at https://target-benchmark.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。