提出STOP算法,让脉冲神经网络更省电且精度更高。
STOP: Spatiotemporal Orthogonal Propagation for Weight-Threshold-Leakage Synergistic Training of Deep Spiking Neural Networks
- 通过时空正交传播,同步优化权重与发放阈值
- 在四个数据集上达到94.84%~98.26%的准确率
- 适合资源受限的边缘智能场景
人工智能物联时代亟需更高能效的边缘计算范式,如基于时空稀疏二进制脉冲的类脑脉冲神经网络(SNN)。然而,缺乏高效高精度的深度SNN学习算法制约其在严格成本限制下的实际部署。本文提出时空正交传播(STOP)算法,在统一的时间前向追踪框架下,实现突触权重、发放阈值与泄漏因子的协同学习,同时避免存储所有时间步神经状态带来的巨大内存开销。其特点是空间反向误差与时间前向追踪正交独立传播,显著降低计算复杂度。在采用VGG-11或ResNet-18结构的深度卷积SNN上,该算法在CIFAR-10、CIFAR-100、DVS-Gesture和DVS-CIFAR10数据集上分别获得94.84%、74.92%、98.26%和77.10%的识别准确率。相比其他深度SNN训练方法,本方案更适用于资源受限但需高精度本地学习的边缘智能场景。
原文摘要 · Abstract (English)
The prevailing of artificial intelligence-of-things calls for higher energy-efficient edge computing paradigms, such as neuromorphic agents leveraging brain-inspired spiking neural network (SNN) models based on spatiotemporally sparse binary spikes. However, the lack of efficient and high-accuracy deep SNN learning algorithms prevents them from practical edge deployments at a strictly bounded cost. In this paper, we propose the spatiotemporal orthogonal propagation (STOP) algorithm to tackle this challenge. Our algorithm enables fully synergistic learning of synaptic weights as well as firing thresholds and leakage factors in spiking neurons to improve SNN accuracy, in a unified temporally-forward trace-based framework to mitigate the huge memory requirement for storing neural states across all time-steps in the forward pass. Characteristically, the spatially-backward neuronal errors and temporally-forward traces propagate orthogonally to and independently of each other, substantially reducing computational complexity. Our STOP algorithm obtained high recognition accuracies of 94.84%, 74.92%, 98.26% and 77.10% on the CIFAR-10, CIFAR-100, DVS-Gesture and DVS-CIFAR10 datasets with adequate deep convolutional SNNs of VGG-11 or ResNet-18 structures. Compared with other deep SNN training algorithms, our method is more plausible for edge intelligent scenarios where resources are limited but high-accuracy in-situ learning is desired.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。