提出在线重排机制,动态修正检索结果相关性,提升大集合搜索召回率。
Breaking the Lens of the Telescope: Online Relevance Estimation over Large Retrieval Sets
- 通过在线强化学习持续更新文档相关性评分,动态调整候选集优先级。
- 在TREC测试中显著提升召回率,尤其在早期被排除的文档上表现优异。
- 适合需要高召回的大规模检索系统,对资源受限场景友好。
先进相关性模型(如大语言模型)虽准确,但计算成本高,难以处理大规模文档集合。现有系统常采用望远镜式检索:先用快速但粗糙的词法与语义检索器筛选候选,再进行精细排序。然而该方法严重依赖初期检索质量,可能导致相关文档被提前排除。本文提出一种新型重排范式——在线相关性估计,可在排序过程中持续更新查询的相关性评分。不同于传统一次性重排前k个文档,该方法迭代重评最可能相关的子集,并利用在线强化学习算法,根据最终模型的估计动态调整剩余文档的优先级。此动态过程可缓解望远镜式系统的召回瓶颈,重新激活早期被判定为不相关甚至完全排除的文档。我们在TREC基准下验证了两种场景:混合检索与自适应检索,实验表明该方法样本效率高,显著提升召回率,验证了其在现代搜索引擎中的有效性。
原文摘要 · Abstract (English)
Advanced relevance models, such as those that use large language models (LLMs), provide highly accurate relevance estimations. However, their computational costs make them infeasible for processing large document corpora. To address this, retrieval systems often employ a telescoping approach, where computationally efficient but less precise lexical and semantic retrievers filter potential candidates for further ranking. However, this approach heavily depends on the quality of early-stage retrieval, which can potentially exclude relevant documents early in the process. In this work, we propose a novel paradigm for re-ranking called online relevance estimation that continuously updates relevance estimates for a query throughout the ranking process. Instead of re-ranking a fixed set of top-k documents in a single step, online relevance estimation iteratively re-scores smaller subsets of the most promising documents while adjusting relevance scores for the remaining pool based on the estimations from the final model using an online bandit-based algorithm. This dynamic process mitigates the recall limitations of telescoping systems by re-prioritizing documents initially deemed less relevant by earlier stages -- including those completely excluded by earlier-stage retrievers. We validate our approach on TREC benchmarks under two scenarios: hybrid retrieval and adaptive retrieval. Experimental results demonstrate that our method is sample-efficient and significantly improves recall, highlighting the effectiveness of our online relevance estimation framework for modern search systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。