arXiv:2410.07485cs.DBcs.LG2024-10被引 3

用高斯混合模型分析数值列分布,生成更精准的嵌入表示。

Gem: Gaussian Mixture Model Embeddings for Numerical Feature Distributions

  • 基于高斯混合模型聚类相似数值分布的列
  • 生成概率矩阵表征列与分布成分的关联性
  • 适合处理数值型数据的类型识别与搜索任务

嵌入技术已广泛应用于实体消歧、数据集搜索和语义类型检测等数据管理任务。尽管数值列普遍存在,但现有方法对数值特征的关注远低于分类数据。本文提出Gem(高斯混合模型嵌入),通过高斯混合模型(GMM)识别并聚类具有相似值分布的列。引入签名机制,为每列生成概率矩阵,表示其属于特定高斯分量的可能性,可用于语义类型判定等应用。进一步生成三类数值属性嵌入:分布、统计与上下文。核心方法仅依赖数值列本身,不使用表名或邻列信息,但可融合属性名等外部证据。在四个基准数据集上,无论纯数值还是含上下文的任务中,Gem均持续优于多个基线方法。

原文摘要 · Abstract (English)

Embeddings are now used to underpin a wide variety of data management tasks, including entity resolution, dataset search and semantic type detection. Such applications often involve datasets with numerical columns, but there has been more emphasis placed on the semantics of categorical data in embeddings than on the distinctive features of numerical data. In this paper, we propose a method called Gem (Gaussian mixture model embeddings) that creates embeddings that build on numerical value distributions from columns. The proposed method specializes a Gaussian Mixture Model (GMM) to identify and cluster columns with similar value distributions. We introduce a signature mechanism that generates a probability matrix for each column, indicating its likelihood of belonging to specific Gaussian components, which can be used for different applications, such as to determine semantic types. Finally, we generate embeddings for three numerical data properties: distributional, statistical, and contextual. Our core method focuses solely on numerical columns without using table names or neighboring columns for context. However, the method can be combined with other types of evidence, and we later integrate attribute names with the Gaussian embeddings to evaluate the method's contribution to improving overall performance. We compare Gem with several baseline methods for numeric only and numeric + context tasks, showing that Gem consistently outperforms the baselines on four benchmark datasets.

数值嵌入高斯混合数据管理分布建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。