arXiv:2412.12807math.STcs.LG2024-12

通过允许模型拒绝不确定判断,实现更低错误率的高风险决策

Ask for More Than Bayes Optimal: A Theory of Indecisions for Selective Hypothesis Testing

  • 用拒绝判断机制,在保证精度前提下减少人工干预次数
  • 在类间分离差时仍能逼近最优准确率,且拒绝样本极少
  • 适用于医疗、金融等需谨慎决策的场景

选择性分类是一种在高风险场景中实现自动化决策的强大工具,允许分类器仅在自信时做出判断,而在不确定性高时拒绝决策。给定目标精度,目标是最小化需要人工干预的不确定样本数量。对于困难问题,若不拒绝判断则无法达到目标精度。通过合理使用拒绝判断,可使误分类率低于贝叶斯误差率,同时最小化总体拒绝比例。本文刻画了选择性分类的最优风险,建立了连续性和单调性性质,支持最优拒绝策略的选择。重新审视基于奈曼-皮尔逊检验框架的选择性推断,其中拒绝判断可在固定第一类错误率下控制第二类错误率。针对分类与检验任务,提出校准方法,并分析了插件分类器的过失风险及基于精度校准带来的额外拒绝量。在二元高斯混合模型中,识别出指数级相变现象,表明即使类别分离较差,极小的拒绝量也能实现近似最优精度。对高斯混合模型和真实数据集的实验展示了拒绝判断如何提升选择性准确率。

原文摘要 · Abstract (English)

Selective classification is a powerful tool for automated decision-making in high-risk scenarios, allowing classifiers to act only when confident and abstain when uncertainty is high. Given a target accuracy, our goal is to minimize the number of indecisions, which are observations that we do not automate. For difficult problems, the target accuracy may be unattainable without abstaining from making a decision. By using indecisions, we can target a misclassification rate below the Bayes error rate, while minimizing overall indecision mass. We provide a characterization of the optimal risk in selective classification, establishing continuity and monotonicity properties that enable optimal indecision selection. We revisit selective inference via the Neyman-Pearson testing framework, where indecision enables control of Type~II error given a fixed Type~I error probability. For both classification and testing, we propose a calibration method, and analyze the excess risk of plug-in classifiers and the excess indecision mass produced by accuracy-based calibration. In the binary Gaussian mixture model, we identify an exponent-level phase transition, showing that minimal indecision can yield near-optimal accuracy even under poor class separation. Experiments on Gaussian mixtures and real datasets illustrate how indecision can improve selective accuracy.

选择性分类贝叶斯误差决策优化高风险应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。