针对小数据迁移学习中的模型冗余问题,提出精度感知剪枝机制防止准确率持续下降。
An accuracy-aware extension to LRP-based pruning for CNNs to prevent cascading accuracy degradation in data-scarce transfer learning
- 基于LRP的逐层重要性评估,动态调整剪枝速率与顺序
- 通过类别准确率调和平均数控制剪枝过程,避免精度级联下降
- 在极少量数据下仍保持高精度,适合资源受限场景
在小数据迁移学习中,预训练的卷积神经网络(CNN)常作为特征提取器构建高精度分类模型。由于数据稀缺,微调困难,通常需固定权重,但大量无关滤波器导致冗余与效率降低。为此,已有基于层间相关性传播(LRP)的方法量化各滤波器对推理结果的贡献,实现不必要滤波器的剪枝。然而,现有方法存在级联精度下降问题。本文提出一种精度感知的剪枝控制机制,通过类别准确率的调和平均值动态调节剪枝速率与顺序,在压缩VGG16模型的同时保持任务特定性能。实验表明,该方法有效缓解了精度级联下降现象,相比传统LRP剪枝方法,使类平均准确率-剪枝率曲线下面积(AUC)提升约15%。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNNs) pre-trained on large-scale datasets such as ImageNet are widely used as feature extractors to construct high-accuracy classification models from scarce data for specific tasks. In such scenarios, fine-tuning the pre-trained CNN is difficult due to data scarcity, necessitating the use of fixed weights. However, when the weights are kept fixed, many filters that do not contribute to the target task remain in the model, leading to unnecessary redundancy and reduced efficiency. Therefore, effective methods are needed to reduce model size by pruning filters that are unnecessary for inference. To address this, approaches utilizing Layer-wise Relevance Propagation (LRP) have been proposed. LRP quantifies the contribution of each filter to the inference result, enabling the pruning of filters with low relevance. However, existing LRP-based pruning methods have been observed to cause cascading accuracy degradation. In this study, we introduce an accuracy-aware pruning control mechanism for existing LRP-based filter pruning methods, which suppresses cascading accuracy degradation by dynamically adjusting the pruning rate and the pruning order using the harmonic mean of class accuracy, and compresses the pre-trained model while preserving task-specific performance in a small-data environment. We demonstrate that this control mechanism effectively mitigates cascading accuracy degradation and achieves higher classification accuracy compared to existing LRP-based pruning methods, improving the class-averaged area under the accuracy-pruning-rate curve (AUC) of VGG16 by approximately 15\% over conventional LRP-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。