通过比例估计提升分布外检测模型选择效果，显著降低误报率。
DSDE: Using Proportion Estimation to Improve Model Selection for Out-of-Distribution Detection
- 用反向p值策略估算模型库中判定为异常的比例
- 在CIFAR10上将误报率从11.07%降至3.31%
- 适合关注检测可靠性与误报控制的研究者
模型库是提升单模型分布外（OoD）检测性能的有效工具，主要通过模型选择与检测器融合实现。然而，现有方法缺乏对模型选择结果的不确定性量化，且集成过程多关注控制真阳性率（TPR），忽视了假阳性率（FPR）。本文强调模型库中将测试样本识别为异常的模型比例所蕴含的关键信息，该比例直接影响检测误差率。为此，我们反向使用常见的序列p值策略，先设定拒绝域再估计误差率，并引入变点检测视角，提出一种具有自动超参数选择能力的比例估计方法，命名为DSDE。在CIFAR10和CIFAR100上的实验表明，该方法有效应对OoD检测挑战。CIFAR10实验显示，相比表现最优的单模型检测器，DSDE将FPR从11.07%降至3.31%。
原文摘要 · Abstract (English)
Model library is an effective tool for improving the performance of single-model Out-of-Distribution (OoD) detector, mainly through model selection and detector fusion. However, existing methods in the literature do not provide uncertainty quantification for model selection results. Additionally, the model ensemble process primarily focuses on controlling the True Positive Rate (TPR) while neglecting the False Positive Rate (FPR). In this paper, we emphasize the significance of the proportion of models in the library that identify the test sample as an OoD sample. This proportion holds crucial information and directly influences the error rate of OoD detection.To address this, we propose inverting the commonly-used sequential p-value strategies. We define the rejection region initially and then estimate the error rate. Furthermore, we introduce a novel perspective from change-point detection and propose an approach for proportion estimation with automatic hyperparameter selection. We name the proposed approach as DOS-Storey-based Detector Ensemble (DSDE). Experimental results on CIFAR10 and CIFAR100 demonstrate the effectiveness of our approach in tackling OoD detection challenges. Specifically, the CIFAR10 experiments show that DSDE reduces the FPR from 11.07% to 3.31% compared to the top-performing single-model detector.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。