用多维注意力自动搜寻脉冲神经网络最优结构，性能更优且功耗更低。
Differentiable architecture search with multi-dimensional attention for spiking neural networks
- 引入多维注意力机制的可微架构搜索方法
- CIFAR10准确率达94.40%，CIFAR100达76.52%
- 模型整体脉冲数稳定在10万级，适合低功耗场景
脉冲神经网络（SNNs）因其低功耗特性在人工智能领域备受关注。然而，现有SNN方法大多直接沿用人工神经网络（ANN）结构，导致性能不理想。为此，本文结合神经架构搜索（NAS）方法，提出多维注意力可微架构搜索（MA-DARTS），直接自动化搜索SNN的最优网络结构。首先构建可微的两级搜索空间，在固定层数下进行微观结构实验；随后引入多维注意力机制，实现该空间内的算法优化。大量实验表明，本方法在参数量相当的情况下，于CIFAR10上达到94.40%准确率，CIFAR100上达76.52%。同时全程监测各单元脉冲数（NoS），结果显示模型整体脉冲数在验证集约110K、训练集约100K，趋于稳定。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) have gained enormous popularity in the field of artificial intelligence due to their low power consumption. However, the majority of SNN methods directly inherit the structure of Artificial Neural Networks (ANN), usually leading to sub-optimal model performance in SNNs. To alleviate this problem, we integrate Neural Architecture Search (NAS) method and propose Multi-Attention Differentiable Architecture Search (MA-DARTS) to directly automate the search for the optimal network structure of SNNs. Initially, we defined a differentiable two-level search space and conducted experiments within micro architecture under a fixed layer. Then, we incorporated a multi-dimensional attention mechanism and implemented the MA-DARTS algorithm in this search space. Comprehensive experiments demonstrate our model achieves state-of-the-art performance on classification compared to other methods under the same parameters with 94.40% accuracy on CIFAR10 dataset and 76.52% accuracy on CIFAR100 dataset. Additionally, we monitored and assessed the number of spikes (NoS) in each cell during the whole experiment. Notably, the number of spikes of the whole model stabilized at approximately 110K in validation and 100k in training on datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。