让预测结果更可能是个单一答案,提升实用性。
Singleton-Optimized Conformal Prediction
- 设计新非一致性评分,专为减少非单例预测集而优化
- 实验显示单例频率提升超20%,平均集合大小几乎不变
- 适合需要明确决策的场景,如医疗诊断或AI问答
置信预测可构造以指定概率覆盖真实结果的预测集,但常导致过大预测集,实践中成本高。最理想的结果是单例预测——明确判断。现有高效方法主要优化平均集合大小。为此,我们提出一种新非一致性评分,旨在最小化生成非单例集合的概率。基于一个非凸约束优化问题,我们提供几何重述及算法,可在O(K)时间内计算K类问题的非一致性评分与相应分割置信预测集。将该评分用于分割置信预测,得到所提出的单例优化置信预测(SOCOP)方法。在图像分类和大模型多选题问答任务上评估,对比标准非一致性评分(如负标签概率估计及其累积分布函数),结果表明SOCOP显著提升单例频率(有时超过20%),对平均集合大小影响极小。
原文摘要 · Abstract (English)
Conformal prediction can be used to construct prediction sets that cover the true outcome with a desired probability, but can sometimes lead to large prediction sets that are costly in practice. The most useful outcome is a singleton prediction-an unambiguous decision-yet existing efficiency-oriented methods primarily optimize average set size. Motivated by this, we propose a new nonconformity score that aims to minimize the probability of producing non-singleton sets. Starting from a non-convex constrained optimization problem as a motivation, we provide a geometric reformulation and associated algorithm for computing the nonconformity score and associated split conformal prediction sets in O(K) time for K-class problems. Using this score in split conformal prediction leads to our proposed Singleton-Optimized Conformal Prediction (SOCOP) method. We evaluate our method in experiments on image classification and LLM multiple-choice question-answering, comparing with standard nonconformity scores such as the (negative) label probability estimates and their cumulative distribution function; both of which are motivated by optimizing length. The results show that SOCOP increases singleton frequency (sometimes by over 20%) compared to the above scores, with minimal impact on average set size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。