让大模型自动匹配数据表字段,还能自我优化,无需人工标注。
Matchmaker: Self-Improving Large Language Model Programs for Schema Matching
- 用生成-精炼-打分三步流程,让大模型理解不同数据结构的对应关系。
- 在医疗数据集上性能超越现有方法,零样本下仍表现优异。
- 适合需要快速整合异构数据的医疗、金融等领域的研究人员。
模式匹配——即在具有不同表格和层级结构的异构数据源间识别属性对应关系——对构建可互操作的机器学习就绪数据至关重要。该问题在医疗、金融和电商等领域影响深远,同时也能提升通用机器学习模型的训练数据规模。然而,由于模式间存在结构与语义异质性,该任务极具挑战性。以往基于机器学习的方法要么依赖大量标注数据(往往不现实),要么零样本性能差。为此,我们提出 Matchmaker:一种用于模式匹配的组合式语言模型程序,包含候选生成、精炼与置信度评分三个模块。其通过一种新颖的优化方法实现零样本自改进,利用合成上下文示例引导语言模型推理。实验证明,在真实医疗模式匹配基准上,Matchmaker 显著优于现有基于机器学习的方法,展现了加速数据集成与机器学习就绪数据互操作性的潜力。
原文摘要 · Abstract (English)
Schema matching -- the task of finding matches between attributes across disparate data sources with different tables and hierarchies -- is critical for creating interoperable machine learning (ML)-ready data. Addressing this fundamental data-centric problem has wide implications, especially in domains like healthcare, finance and e-commerce -- but also has the potential to benefit ML models more generally, by increasing the data available for ML model training. However, schema matching is a challenging ML task due to structural/hierarchical and semantic heterogeneity between different schemas. Previous ML approaches to automate schema matching have either required significant labeled data for model training, which is often unrealistic or suffer from poor zero-shot performance. To this end, we propose Matchmaker - a compositional language model program for schema matching, comprised of candidate generation, refinement and confidence scoring. Matchmaker also self-improves in a zero-shot manner without the need for labeled demonstrations via a novel optimization approach, which constructs synthetic in-context demonstrations to guide the language model's reasoning process. Empirically, we demonstrate on real-world medical schema matching benchmarks that Matchmaker outperforms previous ML-based approaches, highlighting its potential to accelerate data integration and interoperability of ML-ready data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。