用费雪信息筛选关键参数,提升图像分类标注效率
FisherMask: Enhancing Neural Network Labeling Efficiency in Image Classification Using Fisher Information
- 基于费雪信息值掩码重要网络参数,指导主动学习选样
- 在CIFAR-10和FashionMNIST上显著优于现有方法,尤其在数据不平衡时
- 适合需要减少人工标注量的模型训练场景
深度学习模型虽性能优异,但依赖大量标注数据,而人工标注耗时耗力。为此,本文提出FisherMask,一种基于费雪信息的主动学习方法,通过掩码关键网络参数来识别最具影响力的样本。该方法利用费雪信息选择最重要的参数,增强批量主动学习效果,提升样本选择效率。费雪信息具备良好的统计性质,有助于理解模型行为与主动学习流程的性能特征。大量实验表明,FisherMask在多种数据集(如CIFAR-10、FashionMNIST)上均显著优于当前最优方法,尤其在数据分布不均衡条件下表现突出,大幅提升标注效率。其可作为衡量模型参数对数据敏感性的有效工具。代码已开源。
原文摘要 · Abstract (English)
Deep learning (DL) models are popular across various domains due to their remarkable performance and efficiency. However, their effectiveness relies heavily on large amounts of labeled data, which are often time-consuming and labor-intensive to generate manually. To overcome this challenge, it is essential to develop strategies that reduce reliance on extensive labeled data while preserving model performance. In this paper, we propose FisherMask, a Fisher information-based active learning (AL) approach that identifies key network parameters by masking them based on their Fisher information values. FisherMask enhances batch AL by using Fisher information to select the most critical parameters, allowing the identification of the most impactful samples during AL training. Moreover, Fisher information possesses favorable statistical properties, offering valuable insights into model behavior and providing a better understanding of the performance characteristics within the AL pipeline. Our extensive experiments demonstrate that FisherMask significantly outperforms state-of-the-art methods on diverse datasets, including CIFAR-10 and FashionMNIST, especially under imbalanced settings. These improvements lead to substantial gains in labeling efficiency. Hence serving as an effective tool to measure the sensitivity of model parameters to data samples. Our code is available on \url{https://github.com/sgchr273/FisherMask}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。