通过动态超块剪枝，加速稀疏检索，提升效率且不丢精度。
Dynamic Superblock Pruning for Fast Learned Sparse Retrieval
- 将文档块组织成超块，提前判断可剪枝的文档组。
- 在MS MARCO数据集上单线程CPU下提速显著优于现有方法。
- 适合需要快速检索且对结果准确性要求高的应用场景。
本文提出在在线文档检索中对学习到的稀疏表示进行超块剪枝（SP）。SP将稀疏索引组织为一系列文档块的超块，并在访问子块前进行超块级别的选择，判断某些超块是否可被剪枝。该方法超越了以往的平坦块或聚类剪枝，能够提前识别出不可能或不太可能出现在最终前k列表中的文档组。SP可在高相关性竞争约束下以安全或近似方式加速稀疏检索。实验表明，在单线程CPU上的MS MARCO段落数据集上，该方案显著优于当前最先进基线。
原文摘要 · Abstract (English)
This paper proposes superblock pruning (SP) during top-k online document retrieval for learned sparse representations. SP structures the sparse index as a set of superblocks on a sequence of document blocks and conducts a superblock-level selection to decide if some superblocks can be pruned before visiting their child blocks. SP generalizes the previous flat block or cluster-based pruning, allowing the early detection of groups of documents that cannot or are less likely to appear in the final top-k list. SP can accelerate sparse retrieval in a rank-safe or approximate manner under a high-relevance competitiveness constraint. Our experiments show that the proposed scheme significantly outperforms state-of-the-art baselines on MS MARCO passages on a single-threaded CPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。