用元模型集成提升图像分类模型的置信度校准,效果更好且更省参数。
Classifier Ensemble for Efficient Uncertainty Calibration of Deep Neural Networks for Image Classification
- 用元模型构建分类器集成,替代传统校准方法。
- 相比单模型,显著降低ECE和MCE,最大降幅达37.5%。
- 适合追求高可靠性、资源受限的深度学习应用。
本文研究用于图像分类的深度神经网络不确定性校准的新颖分类器集成技术。评估了准确率与校准指标,重点关注期望校准误差(ECE)和最大校准误差(MCE)。实验对比了多种构建简单高效分类器集成的方法,包括多数投票和基于元模型的几种方案。结果表明,尽管当前主流图像分类模型在标准数据集上准确率很高,但普遍存在严重校准误差。基础集成方法如多数投票仅带来有限改善,而基于元模型的集成在所有架构下均持续降低ECE与MCE。其中,规模最大的元模型表现最佳,校准提升最显著,且对准确率影响极小。此外,元模型集成在校准性能上优于传统模型集成,同时参数量大幅减少。与传统后处理校准方法相比,该方法无需额外校准数据集。这些发现证明,所提出的元模型集成是一种高效可靠的校准方案,有助于构建更可信的深度学习系统。
原文摘要 · Abstract (English)
This paper investigates novel classifier ensemble techniques for uncertainty calibration applied to various deep neural networks for image classification. We evaluate both accuracy and calibration metrics, focusing on Expected Calibration Error (ECE) and Maximum Calibration Error (MCE). Our work compares different methods for building simple yet efficient classifier ensembles, including majority voting and several metamodel-based approaches. Our evaluation reveals that while state-of-the-art deep neural networks for image classification achieve high accuracy on standard datasets, they frequently suffer from significant calibration errors. Basic ensemble techniques like majority voting provide modest improvements, while metamodel-based ensembles consistently reduce ECE and MCE across all architectures. Notably, the largest of our compared metamodels demonstrate the most substantial calibration improvements, with minimal impact on accuracy. Moreover, classifier ensembles with metamodels outperform traditional model ensembles in calibration performance, while requiring significantly fewer parameters. In comparison to traditional post-hoc calibration methods, our approach removes the need for a separate calibration dataset. These findings underscore the potential of our proposed metamodel-based classifier ensembles as an efficient and effective approach to improving model calibration, thereby contributing to more reliable deep learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。