用新损失函数提升深度分类模型准确率，效果优于传统方法。
Normalized Conditional Mutual Information Surrogate Loss for Deep Neural Classifiers
- 提出归一化条件互信息损失，替代交叉熵训练模型
- ImageNet上准确率提升2.77%，CAMELYON-17上F1提升8.6%
- 兼容多种架构与批量大小，适合追求高精度的分类任务
本文提出一种新型信息论代理损失——归一化条件互信息（NCMI），作为深度神经网络分类器训练中广泛使用的交叉熵（CE）的即插即用替代方案。研究发现模型的NCMI与其准确率呈负相关。基于此，设计了一种交替优化算法以高效最小化NCMI。在图像识别与全切片影像（WSI）分型基准测试中，使用NCMI训练的模型显著超越现有最优损失函数，计算成本与交叉熵相当。在ImageNet上，相比交叉熵，使用ResNet-50时顶1准确率提升2.77%；在CAMELYON-17数据集上，替换交叉熵后宏平均F1提升8.6%，超过最强基线。性能增益在不同网络架构和批量大小下均保持一致，表明NCMI是交叉熵的一种实用且有竞争力的替代方案。
原文摘要 · Abstract (English)
In this paper, we propose a novel information theoretic surrogate loss; normalized conditional mutual information (NCMI); as a drop in alternative to the de facto cross-entropy (CE) for training deep neural network (DNN) based classifiers. We first observe that the model's NCMI is inversely proportional to its accuracy. Building on this insight, we introduce an alternating algorithm to efficiently minimize the NCMI. Across image recognition and whole-slide imaging (WSI) subtyping benchmarks, NCMI-trained models surpass state of the art losses by substantial margins at a computational cost comparable to that of CE. Notably, on ImageNet, NCMI yields a 2.77% top-1 accuracy improvement with ResNet-50 comparing to the CE; on CAMELYON-17, replacing CE with NCMI improves the macro-F1 by 8.6% over the strongest baseline. Gains are consistent across various architectures and batch sizes, suggesting that NCMI is a practical and competitive alternative to CE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。