让神经网络根据输入难易提前退出，显著降低计算量。
Early-exit Convolutional Neural Networks
- 在CNN中设置多个可提前退出的分支，根据置信度决定是否提前终止推理。
- 在多个数据集上实现与原模型相当准确率，计算量降至20%。
- 适合需要低延迟、低功耗推理的部署场景，如移动端或边缘设备。
本文旨在降低卷积神经网络（CNN）在推理过程中的计算成本。传统方法中，输入数据需经过固定结构的网络处理，但简单样本可在早期阶段完成分类，而传统网络无法利用这一特性。为此，本文提出'Early-exit CNNs'（简称EENets），通过在不同层级设置出口块，根据输入复杂度动态调整计算量。每个出口块包含一个置信度分支和一个Softmax分类分支，两者参数独立可学习。训练时不仅优化分类损失，还引入推理计算成本作为目标。测试时，当某层置信度超过阈值，则停止该样本的推理。该方法可适配如ResNet等主流架构。在MNIST、SVHN、CIFAR10和Tiny-ImageNet上的实验表明，早期退出的ResNet在保持相近精度的同时，将计算量降低至原模型的20%。代码已公开于https://github.com/eksuas/eenets.pytorch。
原文摘要 · Abstract (English)
This paper is aimed at developing a method that reduces the computational cost of convolutional neural networks (CNN) during inference. Conventionally, the input data pass through a fixed neural network architecture. However, easy examples can be classified at early stages of processing and conventional networks do not take this into account. In this paper, we introduce 'Early-exit CNNs', EENets for short, which adapt their computational cost based on the input by stopping the inference process at certain exit locations. In EENets, there are a number of exit blocks each of which consists of a confidence branch and a softmax branch. The confidence branch computes the confidence score of exiting (i.e. stopping the inference process) at that location; while the softmax branch outputs a classification probability vector. Both branches are learnable and their parameters are separate. During training of EENets, in addition to the classical classification loss, the computational cost of inference is taken into account as well. As a result, the network adapts its many confidence branches to the inputs so that less computation is spent for easy examples. Inference works as in conventional feed-forward networks, however, when the output of a confidence branch is larger than a certain threshold, the inference stops for that specific example. The idea of EENets is applicable to available CNN architectures such as ResNets. Through comprehensive experiments on MNIST, SVHN, CIFAR10 and Tiny-ImageNet datasets, we show that early-exit (EE) ResNets achieve similar accuracy with their non-EE versions while reducing the computational cost to 20% of the original. Code is available at https://github.com/eksuas/eenets.pytorch
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。