用假设检验检测分类不确定数据,提升高风险场景可靠性
A method for classification of data with uncertainty using hypothesis testing
- 通过双类型假设检验识别重叠区与分布外数据
- 基于训练数据特征值分布量化不确定性,阈值由α分位数决定
- 无需模型改造,适配医疗、金融等高风险决策场景
二分类任务广泛应用于多个领域。然而,传统分类器在两类分布重叠区域或分布外数据上常产生过度自信的预测,不适用于高风险场景。为此,需量化不确定性并采用考虑不确定性的决策方法。现有方法通常需重采样、改进模型结构或优化阈值。本文提出一种基于两种假设检验的新决策方法,可有效检测属于两类分布重叠区域的数据及未包含在训练数据分布中的分布外数据。通过训练模型获取的特征值经验分布量化不确定性,分类阈值由α分位数和(1−α)分位数确定,显著性水平α可根据具体场景设定。
原文摘要 · Abstract (English)
Binary classification is a task that involves the classification of data into one of two distinct classes. It is widely utilized in various fields. However, conventional classifiers tend to make overconfident predictions for data that belong to overlapping regions of the two class distributions or for data outside the distributions (out-of-distribution data). Therefore, conventional classifiers should not be applied in high-risk fields where classification results can have significant consequences. In order to address this issue, it is necessary to quantify uncertainty and adopt decision-making approaches that take it into account. Many methods have been proposed for this purpose; however, implementing these methods often requires performing resampling, improving the structure or performance of models, and optimizing the thresholds of classifiers. We propose a new decision-making approach using two types of hypothesis testing. This method is capable of detecting ambiguous data that belong to the overlapping regions of two class distributions, as well as out-of-distribution data that are not included in the training data distribution. In addition, we quantify uncertainty using the empirical distribution of feature values derived from the training data obtained through the trained model. The classification threshold is determined by the $α$-quantile and ($1-α$)-quantile, where the significance level $α$ is set according to each specific situation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。