微调大模型提升实体匹配性能,解释信息增强效果显著。
Fine-tuning Large Language Models for Entity Matching
- 用大模型生成解释并加入训练数据,提升匹配能力
- 微调后小模型性能明显提升,大模型表现不一
- 结构化解释有助于泛化到同领域数据,但削弱跨领域迁移
生成式大语言模型(LLMs)在实体匹配任务中展现出高零样本性能和对未见实体的泛化能力,是预训练语言模型的有力替代。现有研究多集中于提示工程与上下文学习,本文探索了微调LLMs用于实体匹配的潜力。我们从两个维度展开:1)训练样本的表示方式,实验不同类型的LLM生成解释对训练集的影响;2)利用LLM进行训练样本的选择与生成。除了源数据集上的匹配性能外,还考察了微调对同领域及其他领域数据集泛化能力的影响。实验表明,微调显著提升了小模型性能,而大模型结果参差不齐;同时,微调增强了同领域泛化能力,但损害了跨领域迁移性能。将结构化解释加入训练集可提升四个模型中的三个,而提出的样本选择与生成方法仅改善了Llama 3.1 8B的表现,却降低了GPT-4o-mini的性能。
原文摘要 · Abstract (English)
Generative large language models (LLMs) are a promising alternative to pre-trained language models for entity matching due to their high zero-shot performance and ability to generalize to unseen entities. Existing research on using LLMs for entity matching has focused on prompt engineering and in-context learning. This paper explores the potential of fine-tuning LLMs for entity matching. We analyze fine-tuning along two dimensions: 1) the representation of training examples, where we experiment with adding different types of LLM-generated explanations to the training set, and 2) the selection and generation of training examples using LLMs. In addition to the matching performance on the source dataset, we investigate how fine-tuning affects the models ability to generalize to other in-domain datasets as well as across topical domains. Our experiments show that fine-tuning significantly improves the performance of the smaller models while the results for the larger models are mixed. Fine-tuning also improves the generalization to in-domain datasets while hurting cross-domain transfer. We show that adding structured explanations to the training set has a positive impact on the performance of three out of four LLMs, while the proposed example selection and generation methods, only improve the performance of Llama 3.1 8B while decreasing the performance of GPT-4o-mini.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。