自适应拓扑稀疏网络在减少99%参数的同时保持高鲁棒性。
Robustness in sparse artificial neural networks trained with adaptive topology
- 每轮训练动态调整稀疏层连接，实现高效压缩。
- 99%稀疏度下仍达与稠密网络相当的分类准确率。
- 对随机断连、对抗攻击等扰动表现稳健，适合资源受限场景。
我们研究了采用自适应拓扑训练的稀疏人工神经网络的鲁棒性。聚焦于一个简单而有效的架构：三个99%稀疏层后接一个稠密层，应用于MNIST和Fashion MNIST图像分类任务。通过在每轮训练间更新稀疏层的拓扑结构，即使权重数量大幅减少，仍实现了具有竞争力的准确率。主要贡献在于对这些网络鲁棒性的深入分析，探索其在随机连接移除、对抗攻击及连接权重随机化等多种扰动下的表现。大量实验表明，自适应拓扑不仅提升效率，还能维持良好鲁棒性。本工作凸显了自适应稀疏网络在构建高效可靠深度学习模型方面的潜力。
原文摘要 · Abstract (English)
We investigate the robustness of sparse artificial neural networks trained with adaptive topology. We focus on a simple yet effective architecture consisting of three sparse layers with 99% sparsity followed by a dense layer, applied to image classification tasks such as MNIST and Fashion MNIST. By updating the topology of the sparse layers between each epoch, we achieve competitive accuracy despite the significantly reduced number of weights. Our primary contribution is a detailed analysis of the robustness of these networks, exploring their performance under various perturbations including random link removal, adversarial attack, and link weight shuffling. Through extensive experiments, we demonstrate that adaptive topology not only enhances efficiency but also maintains robustness. This work highlights the potential of adaptive sparse networks as a promising direction for developing efficient and reliable deep learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。