通过事件注意力机制让神经网络跳过无用信号,提升脉冲序列分类效率。
SkipSNN: Efficiently Classifying Spike Trains with Event-attention
- 引入事件注意力机制,动态过滤噪声,只在关键时刻计算。
- 在多个类脑任务中,比现有SNN模型快2.1倍且准确率更高。
- 适合部署在资源受限设备上的实时脉冲数据处理场景。
脉冲序列分类是机器学习领域的重要课题,每个脉冲序列都是具有信号时空稀疏性与时间噪声特性的二值事件序列。理想的模型应在信号出现时才进行密集计算。因此,脉冲神经网络(SNN)因其对脉冲序列时空稀疏性的天然适配而被广泛采用。然而,传统SNN机制忽略时间噪声问题,导致计算开销大、功耗高,难以在资源受限平台运行。作为事件驱动模型,SNN神经元对任意输入均响应,难以快速定位信号。本文提出事件注意力机制,使SNN能动态突出原始脉冲序列中的有用信号。为此,我们设计了SkipSNN,在原有SNN基础上学习屏蔽噪声,跳过膜电位更新,缩短有效计算图规模。该过程类似人睁眼闭眼筛选信息。我们在多个类脑任务上评估SkipSNN,结果表明其在计算效率和分类精度上均显著优于当前最先进的SNN模型。
原文摘要 · Abstract (English)
Spike train classification has recently become an important topic in the machine learning community, where each spike train is a binary event sequence with \emph{temporal-sparsity of signals of interest} and \emph{temporal-noise} properties. A promising model for it should follow the design principle of performing intensive computation only when signals of interest appear. So such tasks use mainly Spiking Neural Networks (SNNs) due to their consideration of temporal-sparsity of spike trains. However, the basic mechanism of SNNs ignore the temporal-noise issue, which makes them computationally expensive and thus high power consumption for analyzing spike trains on resource-constrained platforms. As an event-driven model, an SNN neuron makes a reaction given any input signals, making it difficult to quickly find signals of interest. In this paper, we introduce an event-attention mechanism that enables SNNs to dynamically highlight useful signals of the original spike trains. To this end, we propose SkipSNN, which extends existing SNN models by learning to mask out noise by skipping membrane potential updates and shortening the effective size of the computational graph. This process is analogous to how people choose to open and close their eyes to filter the information they see. We evaluate SkipSNN on various neuromorphic tasks and demonstrate that it achieves significantly better computational efficiency and classification accuracy than other state-of-the-art SNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。