用领域特定微调提升嵌入模型区分实体真伪的能力。
Domain-Specific Text Embedding Models for Entity Resolution
- 基于领域数据构建三元组,微调预训练嵌入模型以增强身份敏感性。
- 微调后模型在区分真实匹配与高相似假匹配上显著提升性能。
- 适合数据质量管理和信息检索中需要精确实体识别的场景。
通用文本嵌入模型虽能捕捉语义相似性,但难以区分代表同一真实主体的实体记录。本文研究领域特定三元组微调能否使预训练嵌入模型适应身份敏感型检索。构建了包含业务和人物记录的合成数据集,包含保持身份一致的变体及具有挑战性的非匹配样本。评估了两种主流嵌入模型在微调前后基于边界相似度的表现。结果表明,微调显著提升了模型区分真实匹配与高度相似非匹配的能力,证明领域特定三元组训练可有效重塑通用嵌入空间,适用于数据质量管理和信息检索。
原文摘要 · Abstract (English)
General-purpose text embedding models are designed to capture semantic similarity but are not optimised for distinguishing entity records that represent the same real-world business or person. This limitation affects applications such as entity resolution and duplicate record retrieval, where small textual differences may either preserve or change identity. This paper investigates whether domain-specific triplet fine-tuning can adapt pretrained embedding models for identity-sensitive retrieval. A synthetic dataset of business and person records was created with identity-preserving variations and challenging non-matching examples. Two widely used embedding models were evaluated before and after fine-tuning using a margin-based similarity evaluation. The results show substantial improvements in separating true matches from highly similar non-matches, demonstrating that domain-specific triplet training can effectively reshape general-purpose embedding spaces for entity retrieval. These findings suggest that targeted fine-tuning provides a practical approach for improving embedding models in data quality management and information retrieval applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。