为排序算法提供无需假设的置信区间，量化新项目排名的不确定性。
Transductive Conformal Inference for Full Ranking
- 基于共形预测思想，利用已知排名的n个旧项推断未知的m个新项排名。
- 在合成与真实数据上验证，对RankNet、LambdaMart等算法有效提升排名可靠性。
- 首次解决无校准集场景下的排序不确定性问题，适合高风险决策应用。
我们提出一种基于共形预测（Conformal Prediction, CP）的方法，用于量化全排序算法的不确定性。考虑有 $n+m$ 个物品需排序，其中 $n$ 个物品的相对真排名已知，目标是量化算法对剩余 $m$ 个新物品在总排序中的排名误差。由于 $n$ 个原物品的真实排名依赖于 $m$ 个新物品的未知真实排名，无法直接获取校准集以应用经典共形预测方法。为此，我们利用共形p值分布的最新结果，构建未知符合性分数的分布无关上界，进而给出任意物品排名的有效预测集，并控制多个预测集下的误覆盖比例（false coverage proportion）。实验表明，该方法在合成数据和真实数据上对 RankNet、LambdaMart 等先进排序算法均具有良好的有效性。
原文摘要 · Abstract (English)
We introduce a method based on Conformal Prediction (CP) to quantify the uncertainty of full ranking algorithms. We focus on a specific scenario where $n+m$ items are to be ranked by some ``black box'' algorithm. It is assumed that the relative (ground truth) ranking of $n$ of them is known. The objective is then to quantify the error made by the algorithm on the ranks of the $m$ new items among the total $(n+m)$. In such a setting, the true ranks of the $n$ original items in the total $(n+m)$ depend on the (unknown) true ranks of the $m$ new ones. Consequently, we have no direct access to a calibration set to apply a classical CP method. To address this challenge, we propose to construct distribution-free bounds of the unknown conformity scores using recent results on the distribution of conformal p-values. Using these scores upper bounds, we provide valid prediction sets for the rank of any item. We also control the false coverage proportion, a crucial quantity when dealing with multiple prediction sets. Finally, we empirically show on both synthetic and real data the efficiency of our CP method for state-of-the-art algorithms such as RankNet or LambdaMart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。