用稀疏检索引导聚类选择，加速稠密检索响应。
LSTM-based Selective Dense Text Retrieval Guided by Sparse Lexical Retrieval
- 基于聚类的两阶段筛选，用LSTM模型判断相关聚类
- 在MS MARCO和BEIR数据集上速度提升显著，内存开销小
- 适合大规模文本检索系统，尤其对磁盘存储场景友好
本文研究稠密检索与稀疏检索的快速融合，提出一种由稀疏检索引导的聚类式选择性稠密检索方法CluSD。CluSD采用轻量级聚类方法，在两阶段选择过程中利用稀疏检索结果与嵌入聚类的重叠信息，通过LSTM模型快速识别相关聚类，同时仅引入有限的额外内存开销。当需要时，CluSD触发部分稠密检索并执行基于聚类的块式磁盘I/O。本文在内存和磁盘上的MS MARCO与BEIR数据集上评估了CluSD，并与多个基线方法进行了比较。
原文摘要 · Abstract (English)
This paper studies fast fusion of dense retrieval and sparse lexical retrieval, and proposes a cluster-based selective dense retrieval method called CluSD guided by sparse lexical retrieval. CluSD takes a lightweight cluster-based approach and exploits the overlap of sparse retrieval results and embedding clusters in a two-stage selection process with an LSTM model to quickly identify relevant clusters while incurring limited extra memory space overhead. CluSD triggers partial dense retrieval and performs cluster-based block disk I/O if needed. This paper evaluates CluSD and compares it with several baselines for searching in-memory and on-disk MS MARCO and BEIR datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。