解决评估时标签缺失问题，让分类器性能指标更可靠。
Model Evaluation in the Dark: Robust Classifier Metrics with Missing Labels
- 用多重插补法处理评估阶段的缺失标签，提升指标准确性。
- 即使标签缺失非随机（MNAR），预测分布仍保持正确形态。
- 适合在标签不全的真实场景中评估模型性能的研究者使用。
监督学习中的缺失数据问题已有广泛研究，但模型评估阶段的标签缺失问题却常被忽视。忽略含缺失值样本的做法可能引入偏差，尤其在数据缺失非随机（MNAR）情况下。本文提出一种基于多重插补的分类器评估方法，适用于精确率、召回率及ROC-AUC等指标。该方法不仅能提供点估计，还能给出缺失标签下这些指标的预测分布。实验表明，该分布的位置与形状在MNAR情形下仍具合理性。此外，我们证明该分布近似服从高斯分布，并给出了有限样本下的收敛界。进一步的鲁棒性分析证实，在合理误差模型下，该近似仍有效。
原文摘要 · Abstract (English)
Missing data in supervised learning is well-studied, but the specific issue of missing labels during model evaluation has been overlooked. Ignoring samples with missing values, a common solution, can introduce bias, especially when data is Missing Not At Random (MNAR). We propose a multiple imputation technique for evaluating classifiers using metrics such as precision, recall, and ROC-AUC. This method not only offers point estimates but also a predictive distribution for these quantities when labels are missing. We empirically show that the predictive distribution's location and shape are generally correct, even in the MNAR regime. Moreover, we establish that this distribution is approximately Gaussian and provide finite-sample convergence bounds. Additionally, a robustness proof is presented, confirming the validity of the approximation under a realistic error model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。