在标签有噪声时仍能准确预测分类置信区间。
Conformal Prediction of Classifiers with Many Classes based on Noisy Labels
- 基于带噪声标签数据估计无噪声校准阈值。
- 在多类别任务中保证有限样本覆盖率，适用于大规模分类。
- 适合标签不准确的现实场景，如自动标注数据集。
置信预测（Conformal Prediction, CP）通过生成小规模预测集来控制分类系统的不确定性，确保真类别落在该集合内的概率达到预定水平。通常通过模型预测定义得分，并利用验证集设定阈值。本文研究仅拥有带噪声标签校准集时的CP校准问题，提出从噪声标签数据中估计无噪声置信阈值的方法。针对均匀噪声，推导出有限样本覆盖保证，即使在类别数量庞大的任务中依然有效。所提方法称为噪声感知置信预测（NACP）。我们在多个具有大量类别的标准图像分类数据集上验证了其性能。
原文摘要 · Abstract (English)
Conformal Prediction (CP) controls the prediction uncertainty of classification systems by producing a small prediction set, ensuring a predetermined probability that the true class lies within this set. This is commonly done by defining a score, based on the model predictions, and setting a threshold on this score using a validation set. In this study, we address the problem of CP calibration when we only have access to a calibration set with noisy labels. We show how we can estimate the noise-free conformal threshold based on the noisy labeled data. We derive a finite sample coverage guarantee for uniform noise that remains effective even in tasks with a large number of classes. We dub our approach Noise-Aware Conformal Prediction (NACP). We illustrate the performance of the proposed results on several standard image classification datasets with a large number of classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。