通过单滤波器性能评估，实现深度网络高效剪枝而不损失精度。
Advanced deep architecture pruning using single filter performance
- 基于单滤波器性能分析，构建可量化剪枝策略。
- 在VGG-11和EfficientNet-B0上实现高稀疏度剪枝且保持准确率。
- 适用于卷积层与全连接层，适合大模型压缩场景。
神经网络剪枝可降低推理时的计算复杂度、能耗和延迟。近期研究提出一种量化深度学习中各层单个滤波器性能的新方法，并揭示了深度学习的宏观行为如何由每个滤波器的微观表现及其协同作用决定。受统计力学启发，该视角使我们能从微观性能推导出整体网络行为。本文展示如何利用此理解，在不损害整体精度的前提下，对深层架构的卷积层进行高密度稀释，采用应用滤波器簇连接（AFCC）技术。该方法在CIFAR-100上应用于VGG-11和EfficientNet-B0，剪枝效果优于同等剪枝率的其他方法。此外，该技术扩展至单节点性能分析，实现全连接层的高度剪枝，为超参数化人工智能任务的复杂度显著降低提供了可能。
原文摘要 · Abstract (English)
Pruning the parameters and structure of neural networks reduces the computational complexity, energy consumption, and latency during inference. Recently, a novel underlying mechanism for successful deep learning (DL) was presented based on a method that quantitatively measures the single filter performance in each layer of a DL architecture, and a new comprehensive mechanism of how deep learning works was presented. This statistical mechanics inspired viewpoint enables to reveal the macroscopic behavior of the entire network from the microscopic performance of each filter and their cooperative behavior. Herein, we demonstrate how this understanding paves the path to high quenched dilution of the convolutional layers of deep architectures without affecting their overall accuracy using applied filter cluster connections (AFCC). AFCC is exemplified on VGG-11 and EfficientNet-B0 architectures trained on CIFAR-100, and its high pruning outperforms other techniques using the same pruning magnitude. Additionally, this technique is broadened to single nodal performance and highly pruning of fully connected layers, suggesting a possible implementation to considerably reduce the complexity of over-parameterized AI tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。