用频率代替概率，让大模型问答更可靠且有保障。
Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guarantees
- 基于采样频率计算不确定性，无需模型内部信息。
- 在6个模型4个数据集上，准确区分对错预测，AUROC更高。
- 保证错误率不超过设定值，适合医疗等高风险场景使用。
大型语言模型在多项选择题问答中表现优异，但存在幻觉和过度自信等问题，限制其在高风险领域的应用。为解决此问题，本文提出一种黑箱设置下的基于频率的不确定性量化方法，利用置信区间预测（CP）实现可证明的覆盖率保证。该方法对每个输入进行多次独立采样，以最频繁的输出作为参考，计算预测熵（PE）。在六个LLM和四个数据集（MedMCQA、MedQA、MMLU、MMLU-Pro）上的实验表明，基于频率的PE在区分正确与错误预测方面优于基于对数几率的PE，AUROC指标更优。此外，该方法能有效控制用户指定风险水平下的实际漏报率，验证了采样频率可作为黑箱场景下对数几率的可行替代。本工作提供了一种无分布假设、模型无关的可靠不确定性量化框架，在多项选择题问答中实现覆盖率保障，提升了大模型在实际应用中的可信度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable progress in multiple-choice question answering (MCQA), but their inherent unreliability, such as hallucination and overconfidence, limits their application in high-risk domains. To address this, we propose a frequency-based uncertainty quantification method under black-box settings, leveraging conformal prediction (CP) to ensure provable coverage guarantees. Our approach involves multiple independent samplings of the model's output distribution for each input, with the most frequent sample serving as a reference to calculate predictive entropy (PE). Experimental evaluations across six LLMs and four datasets (MedMCQA, MedQA, MMLU, MMLU-Pro) demonstrate that frequency-based PE outperforms logit-based PE in distinguishing between correct and incorrect predictions, as measured by AUROC. Furthermore, the method effectively controls the empirical miscoverage rate under user-specified risk levels, validating that sampling frequency can serve as a viable substitute for logit-based probabilities in black-box scenarios. This work provides a distribution-free model-agnostic framework for reliable uncertainty quantification in MCQA with guaranteed coverage, enhancing the trustworthiness of LLMs in practical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。