通过通道剪枝缓解深层胶囊网络激活值消失问题，提升模型性能。
Mitigating Vanishing Activations in Deep CapsNets Using Channel Pruning
- 针对深层胶囊网络激活值衰减问题，提出基于通道剪枝的新方法。
- 不同剪枝比例下，剪枝后模型准确率优于未剪枝版本。
- 适用于希望提升深层胶囊网络稳定性和可扩展性的研究者。
胶囊网络在学习部件-整体关系和视角不变性方面优于卷积神经网络，这归功于其多维胶囊结构。传统观点认为增加胶囊层数可提升模型性能，但近期研究表明，胶囊网络因深层胶囊中激活值消失而缺乏可扩展性。本文深入研究了深层胶囊网络中的激活值消失问题，构建并评估了多种具有不同胶囊数量、胶囊维度和中间层数量的胶囊网络模型。与传统模型剪枝减少参数量和加速训练不同，本文利用剪枝来缓解深层胶囊层中的激活值消失问题。同时，对主干网络和胶囊层采用不同剪枝比例，以减少无效胶囊数量，最终实现比未剪枝模型更高的准确率。
原文摘要 · Abstract (English)
Capsule Networks outperform Convolutional Neural Networks in learning the part-whole relationships with viewpoint invariance, and the credit goes to their multidimensional capsules. It was assumed that increasing the number of capsule layers in the capsule networks would enhance the model performance. However, recent studies found that Capsule Networks lack scalability due to vanishing activations in the capsules of deeper layers. This paper thoroughly investigates the vanishing activation problem in deep Capsule Networks. To analyze this issue and understand how increasing capsule dimensions can facilitate deeper networks, various Capsule Network models are constructed and evaluated with different numbers of capsules, capsule dimensions, and intermediate layers for this paper. Unlike traditional model pruning, which reduces the number of model parameters and expedites model training, this study uses pruning to mitigate the vanishing activations in the deeper capsule layers. In addition, the backbone network and capsule layers are pruned with different pruning ratios to reduce the number of inactive capsules and achieve better model accuracy than the unpruned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。