用贝叶斯方法自动剪枝神经网络，提升压缩后模型泛化能力。
Efficient Model Compression for Bayesian Neural Networks
- 基于贝叶斯节点包含概率进行剪枝，保留重要连接
- 在多个数据集上剪枝后模型泛化性能更优
- 适合需要轻量化且高鲁棒性的部署场景
模型压缩近年来受到深度学习领域的广泛关注。通过剪枝或丢弃部分输入特征，压缩密集神经网络可降低计算开销，便于部署到存储与内存受限设备，并增强对对抗攻击的抵抗力。本文提出一种新策略，在深度学习框架中模拟贝叶斯模型选择原理。针对使用变分算法训练的全连接贝叶斯神经网络（含spike-and-slab先验），我们计算每个节点的后验包含概率，利用该概率进行剪枝与特征选择。在一系列模拟与真实世界基准数据上的实验表明，所有情况下剪枝后的模型均展现出更好的泛化能力。
原文摘要 · Abstract (English)
Model Compression has drawn much attention within the deep learning community recently. Compressing a dense neural network offers many advantages including lower computation cost, deployability to devices of limited storage and memories, and resistance to adversarial attacks. This may be achieved via weight pruning or fully discarding certain input features. Here we demonstrate a novel strategy to emulate principles of Bayesian model selection in a deep learning setup. Given a fully connected Bayesian neural network with spike-and-slab priors trained via a variational algorithm, we obtain the posterior inclusion probability for every node that typically gets lost. We employ these probabilities for pruning and feature selection on a host of simulated and real-world benchmark data and find evidence of better generalizability of the pruned model in all our experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。