用强化学习方法自动剪掉冗余卷积通道,节省计算量且不损失精度。
Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits
- 将每个特征图视为一个老虎机臂,通过采样评估其移除代价
- UCB1和Thompson采样法在多个数据集上剪掉30%以上通道仍保持原精度
- 适合追求模型轻量化但不想手动调参的工程师使用
卷积神经网络常包含冗余特征图,增加存储与推理开销。本文提出一种基于多臂老虎机的损失感知特征图剪枝框架。特征图剪枝通过移除完整卷积输出通道及其对应滤波器实现,而非孤立权重。每个候选特征图被视为一个臂,在每次试验中临时掩码并采样小批量评估损失变化,再转化为安全移除奖励。固定试验预算后,根据学习得分排序,移除前k个特征图及其滤波器、偏置和下一层输入通道核。研究对比了UCB1与Thompson采样,在LeNet/MNIST上与直接/基准评估法比较,并扩展至MNIST、CIFAR-10、CIFAR-100、SVHN、CUB-200-2011和Oxford Flowers 102。结果表明,两种方法在移除大量特征图的同时,准确率接近原始模型;弗里德曼与内梅尼检验显示UCB1均值排名最高,其次为Thompson采样,两者显著优于贪心与基于幅度的剪枝,且与原模型统计无显著差异。
原文摘要 · Abstract (English)
Convolutional neural networks often contain redundant feature maps that increase storage and inference cost. This paper presents a loss-aware feature-map pruning framework using multi-armed bandits. Feature-map pruning is structured because it removes complete convolutional output channels and their producing filters rather than isolated scalar weights. Each candidate feature map is treated as an arm. At each play time, one map is temporarily masked and evaluated on a sampled mini-batch; the map is then restored and the observed loss change is converted into a safe-removal reward. After a fixed play budget, candidate maps are ranked by learned scores and the top-k maps are permanently removed with their filters, biases and corresponding next-layer input-channel kernels. The study evaluates UCB1 and Thompson Sampling, compares them with direct/oracle-style evaluation on LeNet/MNIST, and extends the evaluation to MNIST, CIFAR-10, CIFAR-100, SVHN, CUB-200-2011 and Oxford Flowers 102. Results show that UCB1 and Thompson Sampling preserve accuracy close to unpruned models while removing feature maps and reducing convolutional computation. Friedman and Nemenyi tests show that UCB1 obtains the highest mean rank, followed by Thompson Sampling; both significantly outperform greedy and magnitude-based pruning while remaining statistically comparable to the original unpruned model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。