无需标签或元数据,用对比学习识别数据集的语义版本。
SAVeD: Semantic Aware Version Discovery
- 通过随机变换生成数据表的增强视图,用自定义编码器嵌入并对比。
- 在五个基准数据集上准确识别未见表格的版本,分离度显著提升。
- 适合数据清洗、版本管理等场景,尤其适用于无标注数据环境。
本文提出SAVeD(语义感知版本发现),一种基于对比学习的框架,用于在不依赖元数据、标签或集成假设的情况下识别结构化数据集的版本。该方法解决了数据科学中因难以识别相似数据集或转换而造成的重复劳动问题。SAVeD采用改进的SimCLR流程,通过随机变换(如行删除、编码扰动)生成数据表的增强视图,使用定制Transformer编码器进行嵌入,并在潜在空间中对比以优化语义相似性。模型学习缩小同一数据集不同视图间的距离,扩大无关表格间的距离。评估指标包括验证准确率和分离度:前者为在保留测试集上正确分类版本/非版本对的比例;后者为版本与非版本表平均相似度之差(由基准定义,不提供给模型)。实验涵盖数据库语义版本化基准中的五个经典数据集,训练后显著提升准确率与分离度,证明其在识别语义修改版本上的能力。相比未经训练的基线及先前最优方法Starmie,SAVeD的自定义编码器达到竞争性或更优表现。
原文摘要 · Abstract (English)
Our work introduces SAVeD (Semantically Aware Version Detection), a contrastive learning-based framework for identifying versions of structured datasets without relying on metadata, labels, or integration-based assumptions. SAVeD addresses a common challenge in data science of repeated labor due to a difficulty of similar work or transformations on datasets. SAVeD employs a modified SimCLR pipeline, generating augmented table views through random transformations (e.g., row deletion, encoding perturbations). These views are embedded via a custom transformer encoder and contrasted in latent space to optimize semantic similarity. Our model learns to minimize distances between augmented views of the same dataset and maximize those between unrelated tables. We evaluate performance using validation accuracy and separation, defined respectively as the proportion of correctly classified version/non-version pairs on a hold-out set, and the difference between average similarities of versioned and non-versioned tables (defined by a benchmark, and not provided to the model). Our experiments span five canonical datasets from the Semantic Versioning in Databases Benchmark, and demonstrate substantial gains post-training. SAVeD achieves significantly higher accuracy on completely unseen tables in, and a significant boost in separation scores, confirming its capability to distinguish semantically altered versions. Compared to untrained baselines and prior state-of-the-art dataset-discovery methods like Starmie, our custom encoder achieves competitive or superior results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。