提出更精确且校准良好的贝叶斯方法，提升类别分布估计的准确性。
Estimating prevalence with precision and accuracy
- 基于贝叶斯框架设计新量化器PQ，兼顾精度与覆盖率。
- 实验验证：分类器判别力、标注数据量和待估数据量影响估计精度。
- 适用于需要可靠不确定性评估的分类系统开发与验证。
与分类任务不同，类别分布估计（或称量化）的目标是估算数据集中各类别的比例。该任务主要面临两个挑战：校正训练集类别分布偏差，以及量化估计的不确定性。现有不确定性量化方法主要包括自助法（bootstrapping）和贝叶斯量化方法，但两者在精度（置信区间宽度）和覆盖率（置信区间校准度）方面的优劣尚不明确。本文提出一种新的贝叶斯量化器Precise Quantifier（PQ），其在精度和覆盖率上均优于现有方法。通过模拟与真实数据集的实验，我们分析了影响量化精度的关键因素：底层分类器的判别能力、训练量化器所用标注数据集的大小，以及待估计类别的无标签数据集规模。研究为量化学习中的不确定性建模提供了深入洞见。
原文摘要 · Abstract (English)
Unlike classification, whose goal is to estimate the class of each data point in a dataset, prevalence estimation or quantification is a task that aims to estimate the distribution of classes in a dataset. The two main tasks in prevalence estimation are to adjust for bias, due to the prevalence in the training dataset, and to quantify the uncertainty in the estimate. The standard methods used to quantify uncertainty in prevalence estimates are bootstrapping and Bayesian quantification methods. It is not clear which approach is ideal in terms of precision (i.e. the width of confidence intervals) and coverage (i.e. the confidence intervals being well-calibrated). Here, we propose Precise Quantifier (PQ), a Bayesian quantifier that is more precise than existing quantifiers and with well-calibrated coverage. We discuss the theory behind PQ and present experiments based on simulated and real-world datasets. Through these experiments, we establish the factors which influence quantification precision: the discriminatory power of the underlying classifier; the size of the labeled dataset used to train the quantifier; and the size of the unlabeled dataset for which prevalence is estimated. Our analysis provides deep insights into uncertainty quantification for quantification learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。