让检索模型更懂查询与文档的关键词重要性
ColBERT-Att: Late-Interaction Meets Attention for Enhanced Retrieval
- 在后期交互框架中引入注意力机制，捕捉词级相似度权重
- 在MS-MARCO、BEIR等数据集上召回率显著提升
- 适合需要精准语义匹配的搜索系统开发者
预训练语言模型生成的向量嵌入是神经信息检索系统的核心组件，广泛应用于各类知识提取任务。ColBERT提出的后期交互范式在保持高精度的同时具备良好的运行效率。然而，现有方法未考虑查询与文档词汇间的注意力权重，而这些权重直观反映了词间相似性的相对重要性，可能有助于更好理解查询与文档的相关性。本文提出ColBERT-Att，将注意力机制显式集成到后期交互框架中，以增强检索性能。实证评估显示，ColBERT-Att在MS-MARCO以及多个BEIR和LoTTE基准数据集上均取得召回率提升。
原文摘要 · Abstract (English)
Vector embeddings from pre-trained language models form a core component in Neural Information Retrieval systems across a multitude of knowledge extraction tasks. The paradigm of late interaction, introduced in ColBERT, demonstrates high accuracy along with runtime efficiency. However, the current formulation fails to take into account the attention weights of query and document terms, which intuitively capture the "importance" of similarities between them, that might lead to a better understanding of relevance between the queries and documents. This work proposes ColBERT-Att, to explicitly integrate attention mechanism into the late interaction framework for enhanced retrieval performance. Empirical evaluation of ColBERT-Att depicts improvements in recall accuracy on MS-MARCO as well as on a wide range of BEIR and LoTTE benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。