arXiv:2412.08194cs.DBcs.LG2024-12被引 46

用小模型找候选、大模型精排,高效准确匹配数据模式

Magneto: Combining Small and Large Language Models for Schema Matching

  • 分两阶段:小模型检索候选,大模型重新排序
  • 在多个领域数据集上达到高精度,且运行时间显著降低
  • 自监督微调小模型,适合需要低成本高效率的场景

近期语言模型的发展为复杂的数据模式匹配任务带来新机遇。现有方法虽证明了语言模型的有效性,但也暴露出重要局限:小语言模型(SLMs)需训练数据(获取成本高),大语言模型(LLMs)则计算开销大,且受限于上下文窗口。本文提出Magneto,一种兼顾成本与精度的模式匹配方案,通过分两阶段的流程——检索与重排序——结合小模型与大模型优势:先用高效的小模型生成候选匹配,再由大模型进行精准重排序,从而在不牺牲准确率的前提下显著降低运行时间。我们提出一种自监督微调小模型的方法,利用大模型生成语法多样的训练数据,并设计有效的提示策略用于重排序。此外,我们与领域专家合作构建了一个新基准,包含真实生物医学数据集,引入了新的挑战。通过在新旧基准上的详尽实验评估,验证了Magneto在不同领域数据集上的可扩展性与高准确性。

原文摘要 · Abstract (English)

Recent advances in language models opened new opportunities to address complex schema matching tasks. Schema matching approaches have been proposed that demonstrate the usefulness of language models, but they have also uncovered important limitations: Small language models (SLMs) require training data (which can be both expensive and challenging to obtain), and large language models (LLMs) often incur high computational costs and must deal with constraints imposed by context windows. We present Magneto, a cost-effective and accurate solution for schema matching that combines the advantages of SLMs and LLMs to address their limitations. By structuring the schema matching pipeline in two phases, retrieval and reranking, Magneto can use computationally efficient SLM-based strategies to derive candidate matches which can then be reranked by LLMs, thus making it possible to reduce runtime without compromising matching accuracy. We propose a self-supervised approach to fine-tune SLMs which uses LLMs to generate syntactically diverse training data, and prompting strategies that are effective for reranking. We also introduce a new benchmark, developed in collaboration with domain experts, which includes real biomedical datasets and presents new challenges to schema matching methods. Through a detailed experimental evaluation, using both our new and existing benchmarks, we show that Magneto is scalable and attains high accuracy for datasets from different domains.

模式匹配小模型大模型数据集成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。