让检索模型动态识别实体，提升知识更新与准确率。
DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
- 引入动态词汇头，融合实体与词元信息进行联合建模。
- 在三个实体丰富数据集上显著超越现有最优基线。
- 适合需要实时知识更新的开放域检索场景。
学习型稀疏检索（LSR）模型采用预训练语言模型的词表，常将实体拆分为无意义片段，降低检索准确率并限制模型对未包含在训练数据中的新知识的吸收能力。本文通过引入维基百科概念与实体，增强LSR词表，使模型能更有效解决歧义并保持知识时效性。核心是动态词汇（DyVo）头，利用已有实体嵌入及实体检索组件识别查询或文档相关实体，生成实体权重并与词元权重融合，形成用于倒排索引高效检索的联合表示。在三个实体丰富的文档排序数据集上的实验表明，所提DyVo模型显著优于当前最佳基线。
原文摘要 · Abstract (English)
Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained transformers, which often split entities into nonsensical fragments. Splitting entities can reduce retrieval accuracy and limits the model's ability to incorporate up-to-date world knowledge not included in the training data. In this work, we enhance the LSR vocabulary with Wikipedia concepts and entities, enabling the model to resolve ambiguities more effectively and stay current with evolving knowledge. Central to our approach is a Dynamic Vocabulary (DyVo) head, which leverages existing entity embeddings and an entity retrieval component that identifies entities relevant to a query or document. We use the DyVo head to generate entity weights, which are then merged with word piece weights to create joint representations for efficient indexing and retrieval using an inverted index. In experiments across three entity-rich document ranking datasets, the resulting DyVo model substantially outperforms state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。