让神经网络自动增删参数，训练更省力、精度更高
Growing Networks with Autonomous Pruning
- 网络边训练边自动增减规模，动态适应数据复杂度
- MNIST仅用6.2千参数达99.44%准确率，CIFAR10用157.8万参数达92.2%
- 适合追求极致轻量模型或研究自适应架构的开发者
本文提出生长网络自主剪枝（GNAP）用于图像分类。与传统卷积神经网络不同，GNAP 在训练过程中动态调整自身规模和参数量，以在满足数据需求的同时尽可能减少参数使用。该方法通过生长与剪枝两种互补机制实现：网络初始参数较少，但在每次收敛后周期性扩展以增强表达能力；在生长阶段之间，模型同时进行分类训练与参数剪枝，完全由梯度下降自主完成。生长阶段提升分类性能，自主剪枝则维持低参数量。在多个图像分类基准测试中，该方法可训练出极稀疏但高精度的神经网络。例如，在MNIST上仅用6.2k参数达到99.44%准确率，在CIFAR10上用157.8k参数达到92.2%准确率。
原文摘要 · Abstract (English)
This paper introduces Growing Networks with Autonomous Pruning (GNAP) for image classification. Unlike traditional convolutional neural networks, GNAP change their size, as well as the number of parameters they are using, during training, in order to best fit the data while trying to use as few parameters as possible. This is achieved through two complementary mechanisms: growth and pruning. GNAP start with few parameters, but their size is expanded periodically during training to add more expressive power each time the network has converged to a saturation point. Between these growing phases, model parameters are trained for classification and pruned simultaneously, with complete autonomy by gradient descent. Growing phases allow GNAP to improve their classification performance, while autonomous pruning allows them to keep as few parameters as possible. Experimental results on several image classification benchmarks show that our approach can train extremely sparse neural networks with high accuracy. For example, on MNIST, we achieved 99.44% accuracy with as few as 6.2k parameters, while on CIFAR10, we achieved 92.2\ accuracy with 157.8k parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。