用带动量的弗兰克-沃尔夫法，少预训练也能剪出更稀疏更准的卷积网络。
Projection-Free CNN Pruning via Frank-Wolfe with Momentum: Sparser Models with Less Pretraining
- 用带动量的弗兰克-沃尔夫算法进行网络剪枝，避免显式投影。
- 仅需2~3轮预训练就达到原模型精度，且模型更稀疏、推理更快。
- 适合追求低计算开销与高压缩率的图像分类部署场景。
我们研究了弗兰克-沃尔夫（Frank-Wolfe, FW）优化方法在卷积神经网络剪枝中的算法变体。这一工作受‘彩票假设’启发，即大型预训练模型中存在性能相当甚至更优的小型子网络。尽管多数相关研究关注深度神经网络整体，本文聚焦于图像分类任务中的卷积神经网络。我们在MNIST数据集上对比了基于权重大小的简单剪枝、标准FW剪枝以及带动量的FW剪枝方法，考察测试准确率、损失、稀疏度和推理时间随密集预训练周期从1到10的变化。结果表明，带动量的FW方法生成的剪枝模型不仅比原始稠密模型更稀疏、准确率更高，也优于简单剪枝基线，且实现中推理开销极小。更重要的是，该方法仅需数个预训练周期即可达成高精度，表明在该设置下无需完整预训练稠密模型。
原文摘要 · Abstract (English)
We investigate algorithmic variants of the Frank-Wolfe (FW) optimization method for pruning convolutional neural networks. This is motivated by the "Lottery Ticket Hypothesis", which suggests the existence of smaller sub-networks within larger pre-trained networks that perform comparatively well (if not better). Whilst most literature in this area focuses on Deep Neural Networks more generally, we specifically consider Convolutional Neural Networks for image classification tasks. Building on the hypothesis, we compare simple magnitude-based pruning, a Frank-Wolfe style pruning scheme, and an FW method with momentum on a CNN trained on MNIST. Our experiments track test accuracy, loss, sparsity, and inference time as we vary the dense pre-training budget from 1 to 10 epochs. We find that FW with momentum yields pruned networks that are both sparser and more accurate than the original dense model and the simple pruning baselines, while incurring minimal inference-time overhead in our implementation. Moreover, FW with momentum reaches these accuracies after only a few epochs of pre-training, indicating that full pre-training of the dense model is not required in this setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。