用熵的变化检测错误标签,提升医疗影像训练数据质量。
On Revisiting Entropy for Identifying Mislabeled Images

- 通过分析预测熵随训练的动态变化,识别错误标签样本。
- 在4个医学影像数据集上优于现有方法,准确率显著提升。
- 适用于多种模型,尤其对CLIP类架构效果突出,易于实现。
训练数据中的错误标签会严重损害深度网络性能,因过参数化模型易记忆错误标签。本文提出一种基于训练动态的新方法,通过观察正确标签样本在训练中熵持续下降,而错误标签样本熵保持较高水平的规律,引入有符号熵积分(SEI)统计量,捕捉预测熵的大小与时间趋势。该方法广泛适用于分类网络,在集成对比语言-图像预训练(CLIP)架构时表现尤为出色。在四个医学影像数据集(涵盖不同模态与病理)上的实验表明,SEI在错误标签识别上达到当前最优性能,同时保持计算高效和实现简单。代码已开源:https://github.com/MedAITech/SEI。
原文摘要 · Abstract (English)
Mislabeled samples in training datasets severely degrade the performance of deep networks, as overparameterized models tend to memorize erroneous labels. We address this challenge by proposing a novel approach for mislabeled data detection that leverages training dynamics. Our method is grounded in the key observation that correctly labeled samples exhibit consistent entropy decrease during training, while mislabeled samples maintain relatively high entropy throughout the training process. Building on this insight, we introduce a signed entropy integral (SEI) statistic that captures both the magnitude and temporal trend of prediction entropy across training epochs. SEI is broadly applicable to classification networks and demonstrates particular effectiveness when integrated with contrastive language-image pretraining (CLIP) architectures. Through extensive experiments on four medical imaging datasets -- a domain particularly susceptible to labeling errors due to diagnostic complexity -- spanning diverse modalities and pathologies, we demonstrate that SEI achieves state-of-the-art performance in mislabeled data identification, outperforming existing methods while maintaining computational efficiency and implementation simplicity. Our code is available at https://github.com/MedAITech/SEI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。