为大模型推荐结果提供可解释的置信度评估，精准定位不可靠排序位置。
EviRank: Evidence-Based Confidence Estimation for LLM-Based Ranking
- 从单次前向传播中提取三类证据，融合生成可靠置信度。
- 在三个数据集上显著提升推荐与不确定性量化性能。
- 适合关注推荐系统可信度与可解释性的研究者和工程师。
大语言模型在推荐系统中展现出潜力，但受限于领域覆盖有限及固有的随机性，其可靠性存疑。现有不确定性量化方法存在两大根本问题：(1) 专为问答设计的全局置信度无法揭示排序列表中哪些位置不可靠；(2) 从模型内部提取的细粒度置信度在所有位置上均呈现低值，难以筛选无效预测。为此，我们提出面向大模型排序的基于证据的置信度估计方法（EviRank）。该方法在一次前向传播中提取三类互补证据，并通过可靠的共识聚合机制进行整合；同时，考虑到排序位置本就不平等，引入位置感知校准；最终，校准后的置信度用于指导排序优化。在三个数据集上的实验表明，该方法在推荐与不确定性量化任务上均达到当前最优性能。
原文摘要 · Abstract (English)
Large Language Models show promise for recommendation, but they raise reliability concerns due to limited domain coverage and inherent stochasticity. Existing uncertainty quantification methods persist two fundamental challenges: (1) the global confidence score designed for question answering fails to reveal which positions are unreliable in ranking list; (2) fine-grained confidence extracted from model internals exhibits uniformly low values across all positions, making it impossible to filter unreliable predictions. To tackle the challenges, we propose an evidence-based confidence estimation for LLM-based ranking (EviRank). We extract three complementary evidences from a single forward pass and aggregate them via reliable opinion aggregation. Furthermore, we recognize that ranking positions are inherently unequal, and introduce a position-aware calibration. Lastly, the calibrated confidence guides ranking optimization. Experiments on three datasets demonstrate that our method achieves state-of-the-art performance on both recommendation and uncertainty quantification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。