用条件互信息剪枝卷积层，模型变小精度几乎不变
Pruning Deep Convolutional Neural Network Using Conditional Mutual Information
- 基于条件互信息衡量特征重要性，逐层筛选关键滤波器
- 在VGG16上剪掉超三分之一滤波器，准确率仅降0.32%
- 支持双向并行剪枝，适合部署在资源受限设备
卷积神经网络在图像分类任务中表现优异，但因模型过大难以部署于资源受限硬件。本文利用互信息分析模型各层对输入特征与输出标签的信息保留能力。提出一种基于条件互信息（CMI）的结构化滤波器剪枝方法，通过矩阵化的Rényi α阶熵数值方法计算各层特征图的重要性，并设计多种CMI公式捕捉跨层特征相关性。结合不同策略确定剪枝阈值，实现前向与后向并行剪枝，显著压缩模型规模同时保持精度。在VGG16与CIFAR-10数据集上的实验表明，该方法可减少超过三分之一的滤波器数量，测试准确率仅下降0.32%。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNNs) achieve high performance in image classification tasks but are challenging to deploy on resource-limited hardware due to their large model sizes. To address this issue, we leverage Mutual Information, a metric that provides valuable insights into how deep learning models retain and process information through measuring the shared information between input features or output labels and network layers. In this study, we propose a structured filter-pruning approach for CNNs that identifies and selectively retains the most informative features in each layer. Our approach successively evaluates each layer by ranking the importance of its feature maps based on Conditional Mutual Information (CMI) values, computed using a matrix-based Renyi α-order entropy numerical method. We propose several formulations of CMI to capture correlation among features across different layers. We then develop various strategies to determine the cutoff point for CMI values to prune unimportant features. This approach allows parallel pruning in both forward and backward directions and significantly reduces model size while preserving accuracy. Tested on the VGG16 architecture with the CIFAR-10 dataset, the proposed method reduces the number of filters by more than a third, with only a 0.32% drop in test accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。