arXiv:2504.14839cs.IRcs.AI2025-04中稿 · SIGIR 2025被引 4

提出新型稀疏化方法,让无推理检索更高效准确。

Exploring $\ell_0$ Sparsification for Inference-free Sparse Retrievers

  • 采用ℓ₀稀疏化思路,优化无推理文档编码过程
  • 在BEIR上达到无推理检索最优性能,媲美顶尖双塔模型
  • 揭示效果与效率的权衡,适合实际部署场景

随着对效率需求的提升,信息检索发展出稀疏检索分支,进一步迈向无推理检索——文档在索引时完成编码,查询无需模型推理。现有稀疏检索模型依赖FLOPS正则化实现稀疏化,但该机制原为双塔编码器设计,应用于不对称的无推理场景时效果不佳。此前针对无推理场景的适配仅限于规则方法,稀疏化潜力未被充分挖掘。本文探索基于ℓ₀的稀疏化方法用于无推理检索器。在BEIR基准上的全面跨域评估显示,本方法在无推理稀疏检索模型中达到当前最佳性能,且与领先双塔稀疏检索模型相当。同时,我们揭示了检索效果与计算效率间的权衡关系,展示了其在真实应用中的实用价值。

原文摘要 · Abstract (English)

With increasing demands for efficiency, information retrieval has developed a branch of sparse retrieval, further advancing towards inference-free retrieval where the documents are encoded during indexing time and there is no model-inference for queries. Existing sparse retrieval models rely on FLOPS regularization for sparsification, while this mechanism was originally designed for Siamese encoders, it is considered to be suboptimal in inference-free scenarios which is asymmetric. Previous attempts to adapt FLOPS for inference-free scenarios have been limited to rule-based methods, leaving the potential of sparsification approaches for inference-free retrieval models largely unexplored. In this paper, we explore $\ell_0$ inspired sparsification manner for inference-free retrievers. Through comprehensive out-of-domain evaluation on the BEIR benchmark, our method achieves state-of-the-art performance among inference-free sparse retrieval models and is comparable to leading Siamese sparse retrieval models. Furthermore, we provide insights into the trade-off between retrieval effectiveness and computational efficiency, demonstrating practical value for real-world applications.

稀疏检索无推理ℓ₀稀疏化BEIR

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。