提升搜索排名阈值估计精度，加速顶K查询处理
Beyond Quantile Methods: Improved Top-K Threshold Estimation for Traditional and Learned Sparse Indexes
- 改进量化方法，通过多阶段优化提高阈值预测准确度
- 新方法将平均低估比例降至接近理想值1.0，显著优于现有技术
- 适用于传统与学习型稀疏索引，对高效检索系统有实用价值
顶K阈值估计旨在预测搜索查询中第k个最高分结果的得分。精确估计可加速多种顶K查询处理算法。现有方法中，量化方法在计算成本适中的情况下表现最佳，采样法在特定场景下也有效。本文提出两项贡献：首先，从量化方法出发，设计一系列改进策略，显著降低常用指标均方低估比例（MUF）；其次，验证所提方法在新型学习型稀疏索引结构上的有效性。最优方案使MUF大幅逼近理想值1.0，代价为略微增加时间和空间开销。
原文摘要 · Abstract (English)
Top-k threshold estimation is the problem of estimating the score of the k-th highest ranking result of a search query. A good estimate can be used to speed up many common top-k query processing algorithms, and thus a number of researchers have recently studied the problem. Among the various approaches that have been proposed, quantile methods appear to give the best estimates overall at modest computational costs, followed by sampling-based methods in certain cases. In this paper, we make two main contributions. First, we study how to get even better estimates than the state of the art. Starting from quantile-based methods, we propose a series of enhancements that give improved estimates in terms of the commonly used mean under-prediction fraction (MUF). Second, we study the threshold estimation problem on recently proposed learned sparse index structures, showing that our methods also work well for these cases. Our best methods substantially narrow the gap between the state of the art and the ideal MUF of 1.0, at some additional cost in time and space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。