arXiv:2505.01452cs.IR2025-05被引 9

提出无需推理的稀疏检索新方法,提升效率与效果。

Effective Inference-Free Retrieval for Learned Sparse Representations

  • 训练时学习每个词的得分,查询编码变查表操作
  • 在MS MARCO上mRR@10提升1点,BEIR上nDCG@10提升1.8点
  • 适合追求高效高精度检索的系统开发者

学习稀疏检索(LSR)利用预训练语言模型将文本编码为学习到的词袋表示。研究表明,稀疏性是实现查询处理器效率与效果平衡的关键。传统方法通过正则化手段控制稀疏度。近期基于倒排索引的高效检索引擎出现,引发疑问:正则化的作用是否改变?本文扩展评估了多种正则化方法在效率、效果及跨域泛化能力上的表现。结果表明,可放松正则化以获得更优编码器;同时发现,查询编码已成为整体性能瓶颈。为此,我们提出学习型无推理检索(Li-LSR),在训练阶段为每个词学习得分,将查询编码转化为无缝查表操作。该方法在域内和域外评估中均达到当前最佳效果,在MS MARCO上mRR@10超越Splade-v3-Doc 1点,在BEIR上nDCG@10提升1.8点。

原文摘要 · Abstract (English)

Learned Sparse Retrieval (LSR) is an effective IR approach that exploits pre-trained language models for encoding text into a learned bag of words. Several efforts in the literature have shown that sparsity is key to enabling a good trade-off between the efficiency and effectiveness of the query processor. To induce the right degree of sparsity, researchers typically use regularization techniques when training LSR models. Recently, new efficient -- inverted index-based -- retrieval engines have been proposed, leading to a natural question: has the role of regularization changed in training LSR models? In this paper, we conduct an extended evaluation of regularization approaches for LSR where we discuss their effectiveness, efficiency, and out-of-domain generalization capabilities. We first show that regularization can be relaxed to produce more effective LSR encoders. We also show that query encoding is now the bottleneck limiting the overall query processor performance. To remove this bottleneck, we advance the state-of-the-art of inference-free LSR by proposing Learned Inference-free Retrieval (Li-LSR). At training time, Li-LSR learns a score for each token, casting the query encoding step into a seamless table lookup. Our approach yields state-of-the-art effectiveness for both in-domain and out-of-domain evaluation, surpassing Splade-v3-Doc by 1 point of mRR@10 on MS MARCO and 1.8 points of nDCG@10 on BEIR.

稀疏检索无推理信息检索高效编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。