剖析大模型实体匹配性能差异,发现模型变体比规模更重要
Beyond Scale and Generation: Understanding Language Model-based Entity Matching
- 控制变量对比三种匹配架构,分离模型大小与变体影响
- 嵌入型模型变体显著提升双编码器性能,生成式仅在分布外优势明显
- 大模型更依赖捷径学习,跨数据集迁移能力成关键考察点
实体匹配旨在识别指向同一真实世界实体的记录。语言模型可通过双编码器、交叉编码器和生成式匹配器架构适配该任务。但以往研究常将匹配器架构与模型主干、模型变体(反映不同预训练目标)及模型规模混淆,难以分离性能提升来源。本文通过受控因子实验,涵盖三种匹配器架构、三种来自Qwen3家族的模型变体和三种模型规模,覆盖九个数据集,共执行1,215次微调。还评估了跨数据集可迁移性和计算成本。结果表明:模型变体对双编码器至关重要——嵌入导向型变体提供更强初始化与更优表征几何,预测下游匹配性能;交叉编码器因联合编码记录对而持续优于双编码器,尽管更大模型部分缩小差距;生成式匹配器未普遍优于交叉编码器,其优势集中在分布外场景,包括记录模式的细微未见差异与跨数据集迁移。此外,更大模型更依赖捷径学习,未必表现更好。这些发现厘清了匹配器架构间性能差异的根本因素,推动未来研究与基准设计应更好解耦架构选择与模型层面因素,并显式评估分布偏移与跨数据集迁移能力。实验结果、代码、训练脚本与评估数据已开源至https://github.com/Jantory/llm-trained-matcher。
原文摘要 · Abstract (English)
Entity matching identifies records that refer to the same real-world entity. Language models can be adapted to this task through bi-encoder, cross-encoder, and generative matcher architectures. However, prior studies often conflate matcher architecture with differences in model backbone, model variant(reflecting different pretraining objectives), and model size, making it difficult to isolate the sources of performance gains. We address this issue through a controlled factorial study spanning three matcher architectures, three model variants and three model sizes from the Qwen3 family, and nine datasets, totaling 1,215 fine-tuning runs. We also evaluate cross-dataset transferability and computational cost. Our results show that model variant is critical for bi-encoders: embedding-oriented variants provide stronger initialization and more favorable representation geometry predictive of downstream matching performance. Cross-encoders retain a consistent advantage over bi-encoders because they jointly encode record pairs rather than representing each record independently, although larger models partially narrow this gap. Generative matchers do not universally outperform cross-encoders. Instead, their advantages concentrate under distribution shift, including subtle unseen differences in record schemas and cross-dataset transfer. We further find that larger models rely more heavily on shortcut learning and therefore do not necessarily perform better. These findings clarify the factors underlying performance differences across matcher architectures and motivate future research and benchmark designs that better disentangle architectural choices from model-level factors while explicitly evaluating distribution shift and cross-dataset transferability. We release our experimental results, code, training scripts, and evaluation data at https://github.com/Jantory/llm-trained-matcher.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。