用置信度校准提升无人机图像分类的可靠性,适合小数据场景
Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Prediction
- 基于预训练模型+少量标注数据生成带置信区间的预测集
- 即使标签少,也能保证95%真实标签覆盖率,平均预测集大小仅2.3个
- 适合边缘设备部署,尤其关注不确定性的实际应用者
本文针对复杂无约束环境下稀疏数据的航空图像分类问题,系统评估了置信度校准方法的性能。研究使用MobileNet、DenseNet和ResNet等预训练模型,在有限标注数据下微调并生成预测集。通过引入温度缩放与不缩放两种校准路径,评估了经验覆盖度与平均预测集大小两个关键指标。结果表明,即使样本量小且非一致性评分简单,置信度校准仍可提供可靠的不确定性估计;温度缩放虽常见,但并不总能缩小预测集。此外,模型压缩技术在该框架中展现出显著潜力,为资源受限环境下的部署提供了可行路径。研究建议未来应深入探索噪声标签影响及高效模型压缩策略。
原文摘要 · Abstract (English)
This paper presents a comprehensive empirical analysis of conformal prediction methods on a challenging aerial image dataset featuring diverse events in unconstrained environments. Conformal prediction is a powerful post-hoc technique that takes the output of any classifier and transforms it into a set of likely labels, providing a statistical guarantee on the coverage of the true label. Unlike evaluations on standard benchmarks, our study addresses the complexities of data-scarce and highly variable real-world settings. We investigate the effectiveness of leveraging pretrained models (MobileNet, DenseNet, and ResNet), fine-tuned with limited labeled data, to generate informative prediction sets. To further evaluate the impact of calibration, we consider two parallel pipelines (with and without temperature scaling) and assess performance using two key metrics: empirical coverage and average prediction set size. This setup allows us to systematically examine how calibration choices influence the trade-off between reliability and efficiency. Our findings demonstrate that even with relatively small labeled samples and simple nonconformity scores, conformal prediction can yield valuable uncertainty estimates for complex tasks. Moreover, our analysis reveals that while temperature scaling is often employed for calibration, it does not consistently lead to smaller prediction sets, underscoring the importance of careful consideration in its application. Furthermore, our results highlight the significant potential of model compression techniques within the conformal prediction pipeline for deployment in resource-constrained environments. Based on our observations, we advocate for future research to delve into the impact of noisy or ambiguous labels on conformal prediction performance and to explore effective model reduction strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。