arXiv:2411.12127cs.LGcs.IT2024-11被引 1

通过类别碰撞率量化分类不确定性,实现更精细的置信度评估。

Fine-Grained Uncertainty Quantification via Collisions

  • 用同一输入出现在不同类中的频率定义碰撞矩阵,衡量类间区分难度。
  • 在多个数据集上验证了碰撞矩阵估计与后验概率推断的有效性。
  • 适合关注模型不确定性和分类边界分析的研究者使用。

我们提出一种新的、直观的类别不确定性量化(UQ)指标——类别碰撞频率,即同一输入被观测到属于不同类的情况。利用碰撞频率构建碰撞矩阵 $S$($K imes K$),这是一种细粒度的不确定性度量,反映 $K$ 个类别中每对类别之间的区分难度。我们探讨了该矩阵的应用,建立了其基本数学性质,并揭示其与贝叶斯误差率(BER)等现有方法的关系。针对仅有一热标签数据的场景,我们提出创新技术估计 $S$:首先训练一个成对对比模型,判断两个输入是否同属一类;证明该模型(可实现PAC学习)能估计 $S$ 的行格拉姆矩阵 $G=SS^T$;最后,在合理假设下,证明 $G$ 可唯一恢复 $S$,这是关于非负矩阵的新结果,可能具有独立价值。建立 $S$ 的估计方法后,我们展示了如何结合对比模型估计任意点的后验类别概率分布。实验在多个数据集上验证了碰撞矩阵与后验分布估计的有效性。

原文摘要 · Abstract (English)

We propose a new and intuitive metric for aleatoric uncertainty quantification (UQ), the prevalence of class collisions defined as the same input being observed in different classes. We use the rate of class collisions to define the collision matrix, a novel and uniquely fine-grained measure of uncertainty. For a classification problem involving $K$ classes, the $K\times K$ collision matrix $S$ measures the inherent difficulty in distinguishing between each pair of classes. We discuss several applications of the collision matrix, establish its fundamental mathematical properties, and show its relationship with existing UQ methods, including the Bayes error rate (BER). We also address the new problem of estimating the collision matrix using one-hot labeled data by proposing a series of innovative techniques to estimate $S$. First, we learn a pair-wise contrastive model which accepts two inputs and determines if they belong to the same class. We then show that this contrastive model (which is PAC learnable) can be used to estimate the row Gramian matrix of $S$, defined as $G=SS^T$. Finally, we show that under reasonable assumptions, $G$ can be used to uniquely recover $S$, a new result on non-negative matrices which could be of independent interest. With a method to estimate $S$ established, we demonstrate how this estimate of $S$, in conjunction with the contrastive model, can be used to estimate the posterior class probability distribution of any point. Experimental results are also presented to validate our methods of estimating the collision matrix and class posterior distributions on several datasets.

不确定性量化分类性能对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。