用大模型选实体,不训练检索器也能达到顶尖效果。
Select, Don't Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selection

- 用大模型做选择,无需训练检索器,直接用BM25检索候选。
- 在ZELDA数据集上微调前的准确率提升至86.3,比之前高4个百分点。
- 能主动拒绝不确定结果,适合对可靠性要求高的应用。
实体消歧(ED)是构建和使用知识图谱的关键任务。当前主流神经方法将ED视为单一任务,但其实包含两个不同子问题:候选实体检索与上下文下的正确实体选择。双编码器模型在同一嵌入空间中同时优化两者,导致表示需在高召回率检索与细粒度选择间权衡,且依赖需持续维护的训练检索器。尽管已有研究将检索器与大语言模型(LLM)结合,但两阶段的相互作用尚未系统研究。本文系统比较了在共享LLM选择阶段下的多种检索策略,包括稀疏检索(BM25)、Web KB搜索及最先进的训练过的密集检索器,搭配多个开源与闭源大模型。结果表明,一旦选择由强大LLM承担,训练检索器带来的增益有限:仅用训练免费的BM25检索器配合LLM选择,就在ZELDA基准上达到新最优表现,inKB micro-F1从82.3提升至86.3(+4);若搭配训练过的密集检索器,进一步达88.5。解耦检索与选择也暴露出现有系统的缺陷:当正确实体未被检索到时,系统被迫预测错误实体。而本框架可在检测到检索失败时选择不回答。在奖励正确拒绝的评估设置下,该训练免费方案达到90.7 F1。
原文摘要 · Abstract (English)
Entity Disambiguation (ED) is a key task for constructing and using knowledge graphs. State-of-the-art neural approaches commonly model ED as a single task, although it consists of two distinct subproblems: retrieving candidate entities and selecting the correct one given context. Dual-encoder models optimize for both within a shared embedding space, forcing representations to balance high-recall retrieval with fine-grained selection, and they require trained retrievers, which are costly to maintain as knowledge graphs change. While recent work has begun to combine retrievers with LLM-based selectors, the interplay between the two stages has not been studied systematically. In this paper, we present a systematic comparison of retrieval strategies for candidate generation under a shared LLM-based selection stage, combining sparse retrieval (BM25), Web KB search, and a state-of-the-art trained dense retriever with several open- and closed-source LLMs. We show that, once selection is delegated to a capable LLM, training the retriever provides only modest additional value: a fully training-free BM25 retriever paired with an LLM selector reaches a new state of the art on the ZELDA benchmark, raising inKB micro-F1 from 82.3 to 86.3 (+4); pairing the same LLM with a trained dense retriever reaches 88.5. Decoupling retrieval from selection also exposes a limitation of current ED systems: when the correct entity is missing from retrieved candidates, they are forced to predict an incorrect entity. In contrast, our framework allows for abstention when retrieval failure is detected. In an evaluation setting that rewards correct abstentions, the training-free BM25 + LLM pipeline reaches 90.7 F1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。