arXiv:2506.04006cs.DBcs.AI2025-06被引 1

通过传递一致性检测多源数据中的实体匹配假阳性,无需人工标注。

TransClean: Finding False Positives in Multi-Source Entity Matching under Real-World Conditions via Transitive Consistency

  • 利用匹配结果的传递一致性自动评估匹配质量
  • 迭代剔除假阳性,平均提升24.42点F1分数
  • 适用于噪声大、分布漂移的真实场景,适合数据清洗任务

我们提出TransClean,一种在真实世界条件下检测多源实体匹配假阳性预测的方法,该条件特征为大规模、高噪声且未标注的多源数据集,并存在分布偏移。TransClean专为高效、鲁棒、快速处理多源数据设计,能应对边缘情况,且仅需少量人工标注。它利用匹配的传递一致性——基于模型对直接评估记录对和隐含记录对的预测,衡量匹配结果的一致性。TransClean通过逐步移除假阳性匹配而尽可能保留真阳性,在每一步中仅通过模型评估估算传递一致性,生成可作为真/假阳性数量代理的指标,无需人工标注即可估计匹配质量并指示可能包含假阳性的记录组。实验表明,将TransClean与基础的配对匹配模型(DistilBERT)或先进端到端方法(CLER)结合,均能有效检测多数假阳性,相比传统配对匹配算法,在多源设置下平均提升24.42 F1分数。

原文摘要 · Abstract (English)

We present TransClean, a method for detecting false positive predictions of entity matching algorithms under real-world conditions characterized by large-scale, noisy, and unlabeled multi-source datasets that undergo distributional shifts. TransClean is explicitly designed to operate with multiple data sources in an efficient, robust and fast manner while accounting for edge cases and requiring limited manual labeling. TransClean leverages the Transitive Consistency of a matching, a measure of the consistency of a pairwise matching model f_theta on the matching it produces G_f_theta, based both on its predictions on directly evaluated record pairs and its predictions on implied record pairs. TransClean iteratively modifies a matching through gradually removing false positive matches while removing as few true positive matches as possible. In each of these steps, the estimation of the Transitive Consistency is exclusively done through model evaluations and produces quantities that can be used as proxies of the amounts of true and false positives in the matching while not requiring any manual labeling, producing an estimate of the quality of the matching and indicating which record groups are likely to contain false positives. In our experiments, we compare combining TransClean with a naively trained pairwise matching model (DistilBERT) and with a state-of-the-art end-to-end matching method (CLER) and illustrate the flexibility of TransClean in being able to detect most of the false positives of either setup across a variety of datasets. Our experiments show that TransClean induces an average +24.42 F1 score improvement for entity matching in a multi-source setting when compared to traditional pair-wise matching algorithms.

实体匹配数据清洗无监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。