用大模型从表格中自动推导出实体类型与关系,解决数据不一致问题。
Conceptual Schema Inference for Tabular Datasets using Large Language Models
- 基于列名和单元格内容,用大模型分析语义推导类型与属性
- 生成的模式结构紧凑且高质量,可扩展至大规模数据仓库
- 适合数据治理、数据库设计人员快速理解异构表格数据
来自数据湖、网页表格和开放数据门户的大量表格数据常源于不同来源,导致表示不一致。理解并组织此类数据仓库仍是重大挑战。以往工作多关注数据集发现与探索,本文提出互补性问题:概念模式推断——直接从原始表格中自动推导出包含实体类型、属性及类型间关系的概念模式。我们提出两种基于大语言模型(LLM)的方法:GeSI利用生成式LLM从表级与列级语义推导层次化类型及其属性,并整合为包含跨类型关系的全局模式;EmSI则通过LLM生成的表嵌入,按列级语义聚类表格,推断每组内属性,并从共享属性模式构建层次结构。实验表明,所提方法在推导模式的简洁性与结构质量上表现优异,具备处理大规模数据仓库的可扩展性,并通过案例研究展示了端到端模式推断的有效性。
原文摘要 · Abstract (English)
Large collections of tabular data from data lakes, web tables and open data portals often originate from heterogeneous sources, leading to representational inconsistencies. Understanding and organizing such repositories therefore remains a major challenge. While prior work has primarily focused on dataset discovery and exploration, this paper addresses the complementary problem of conceptual schema inference: automatically deriving a conceptual schema that captures entity types, attributes and inter-type relationships directly from raw tables. We propose two large language model (LLM)-based approaches that use only column headers and cell values: GeSI uses generative LLMs to infer hierarchical types and their attributes from table- and column-level semantics, and to integrate them into a global schema that also captures relationships across types; EmSI employs LLM-based table embeddings to group tables by column-level semantics, infer attributes within each group, and construct hierarchical structures from shared attribute patterns. Finally, we report an experimental analysis demonstrating the effectiveness of our approaches in terms of the conciseness and structural quality of the inferred schema components, their scalability to large repositories, and a case study illustrating end-to-end schema inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。