根据文档相关性不确定性动态调整计算量,提升重排效率与精度
AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking
- 基于贝叶斯真技能模型估算文档相关性不确定性,自适应决定计算范围和强度
- 在TREC-DL和BEIR上相比固定计算方法,准确率更高且计算开销更低
- 适合需要高效高精度重排的场景,尤其适用于大模型长上下文任务
基于大语言模型(LLMs)的列表级重排能提升检索应用中的前列结果质量。受限于上下文长度和长上下文推理成本,重排通常在固定大小的小子集上进行,最终排名由这些部分结果聚合而成。这种固定计算忽视了查询难度和文档分布,造成效率低下。我们提出AcuRank,一种自适应重排框架,依据文档相关性的不确定性估计动态调整计算量与目标。通过贝叶斯真技能模型,迭代优化相关性估计直至达到足够置信度;显式建模排序不确定性,实现对重排行为的合理控制,并避免对已有置信预测的无谓更新。在TREC-DL和BEIR基准上的实验表明,该方法在准确率-效率权衡上持续优于固定计算基线,且随计算资源增加具有更好扩展性。结果验证了该方法在多样化检索任务和基于LLM的重排模型中的有效性与通用性。
原文摘要 · Abstract (English)
Listwise reranking with large language models (LLMs) enhances top-ranked results in retrieval-based applications. Due to the limit in context size and high inference cost of long context, reranking is typically performed over a fixed size of small subsets, with the final ranking aggregated from these partial results. This fixed computation disregards query difficulty and document distribution, leading to inefficiencies. We propose AcuRank, an adaptive reranking framework that dynamically adjusts both the amount and target of computation based on uncertainty estimates over document relevance. Using a Bayesian TrueSkill model, we iteratively refine relevance estimates until reaching sufficient confidence levels, and our explicit modeling of ranking uncertainty enables principled control over reranking behavior and avoids unnecessary updates to confident predictions. Results on the TREC-DL and BEIR benchmarks show that our method consistently achieves a superior accuracy-efficiency trade-off and scales better with compute than fixed-computation baselines. These results highlight the effectiveness and generalizability of our method across diverse retrieval tasks and LLM-based reranking models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。