研究剪枝对GoogLeNet性能与可解释性的影响，发现重训练后模型性能可超越原版。
Investigating the Effect of Network Pruning on Performance and Interpretability
- 对比无结构、结构化剪枝及输入权重稀疏化方法
- 充分重训练后准确率接近甚至超过原始模型
- 可解释性评分与剪枝率无关，高分未必对应直观理解
深度神经网络通常参数冗余，可通过删除权重进行压缩，这一过程称为剪枝。本文系统研究了不同剪枝技术对GoogLeNet分类性能和可解释性的影响。采用无结构剪枝、结构化剪枝以及连接稀疏化（输入权重剪枝）方法，并在ImageNet验证集上分析网络表现。同时比较了迭代剪枝与单次剪枝等不同重训练策略。结果表明，在足够多的重训练周期下，剪枝后模型性能可逼近甚至超越原始GoogLeNet。为评估可解释性，采用Zimmermann等人提出的机制可解释性评分（MIS）。实验发现，使用MIS作为度量时，可解释性与剪枝率之间无显著关联；且极低准确率的模型仍可能获得高MIS分数，暗示MIS可能不总是符合人类对正确决策依据的理解。
原文摘要 · Abstract (English)
Deep Neural Networks (DNNs) are often over-parameterized for their tasks and can be compressed quite drastically by removing weights, a process called pruning. We investigate the impact of different pruning techniques on the classification performance and interpretability of GoogLeNet. We systematically apply unstructured and structured pruning, as well as connection sparsity (pruning of input weights) methods to the network and analyze the outcomes regarding the network's performance on the validation set of ImageNet. We also compare different retraining strategies, such as iterative pruning and one-shot pruning. We find that with sufficient retraining epochs, the performance of the networks can approximate the performance of the default GoogLeNet - and even surpass it in some cases. To assess interpretability, we employ the Mechanistic Interpretability Score (MIS) developed by Zimmermann et al. . Our experiments reveal that there is no significant relationship between interpretability and pruning rate when using MIS as a measure. Additionally, we observe that networks with extremely low accuracy can still achieve high MIS scores, suggesting that the MIS may not always align with intuitive notions of interpretability, such as understanding the basis of correct decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。