实证发现实体消歧系统需多算法并行、精度召回分治、合并链主动验证
Entity Resolution in Practice: Lessons from a Self-Serve Pipeline
- 多算法并行训练，自动筛选最优匹配方案
- 精度靠规则过滤，召回靠多样化候选检索
- 跨组合并必须人工复核，防错连链式扩散
我们在六个基准数据集（864至500万条记录）上构建并评估了一个自服务实体消歧（ER）系统，总结出三条现有文献中缺失的实践教训：(1) 无单一匹配算法在所有数据集上表现最优——自服务管道无法预判下个数据集特性，建议为每个数据集训练多个算法族，并通过自动比选确定最佳方案；(2) 精度与召回需分别优化——精度依赖硬规则过滤错误匹配，召回则需更丰富的候选生成策略；(3) 一个误链接可隐性合并无关实体——若假设“A匹配B”且“B匹配C”即推出“A匹配C”，单个错误链接可导致数百条记录被错误关联，因此每次跨组合并都必须主动复核。我们希望这些经验能帮助从业者避免我们曾经历的数月无效实验。
原文摘要 · Abstract (English)
We built and evaluated a self-serve entity resolution (ER) system on six benchmarks spanning 864 to 5M records, and three lessons emerged that are absent from existing ER literature. (1) No single matching algorithm wins everywhere - a self-serve pipeline cannot predict its next dataset, so we recommend training several algorithm families per dataset and letting an automatic bake-off pick the winner. (2) Precision and recall need separate fixes, not a shared threshold - precision needs hard rule-based vetoes, recall needs more diverse candidate retrieval. (3) One false-positive link can silently merge unrelated entities - assuming "A matches B" and "B matches C" implies "A matches C" lets a single bad link chain hundreds of records together, so every cross-group merge must be actively re-verified. We hope these lessons save practitioners the months of dead-end experiments that led us to them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。