提出XOResNet,用异或残差和新捷径结构提升脉冲神经网络性能
XOResNet: Exclusive-OR Meta-Residuals Facilitate Deep Spiking Neural Networks Learning

- 设计OA捷径融合双分支输出,减少脉冲冗余与信息丢失
- 引入异或选择预学习残差,避免主干分支重复学习
- 在多个数据集上超越现有最优脉冲网络,适合神经形态计算研究者
脉冲神经网络(SNNs)在深度模型中展现出优异的学习与表征能力。受ResNet成功启发,尝试构建深层SNN的残差学习架构。然而,现有方法仍存在身份映射中脉冲冗余、非身份映射中信息丢失及主干分支冗余学习等问题。本文首先针对上述问题,提出一种OR-ADD(OA)捷径连接,用于合并残差结构中两分支的脉冲/电流输出。为进一步缓解主干分支的冗余学习,引入异或(XOR)元残差概念,通过异或操作筛选预学习残差注入主干分支。结合OA捷径与XOR元残差,构建XOR残差模块,并基于此搭建不同深度的XOResNet。在Fashion-MNIST、CIFAR-10、CIFAR-100和miniImageNet四个数据集上的大量实验表明,XOResNet优于现有基于梯度优化的最先进深层SNN,验证了所提组件在克服SNN残差学习根本缺陷方面的有效性,为构建高性能神经形态系统提供了新架构思路。
原文摘要 · Abstract (English)
Spiking neural networks (SNNs) hold promise for demonstrating superior learning and representation capabilities in deep models. Given the tremendous success of ResNet in deep learning, it would naturally follow to train deep SNNs with residual learning. However, existing residual structures for constructing deep SNNs still present challenges of spike redundancy or information loss, as well as redundant learning. In the present study, we first aim to address issues of relative spike redundancy in identity mapping and information loss in non-identity mapping. To this end, we propose an OR-ADD (OA) shortcut connection to merge output spikes/currents from two branches in the residual structure. Furthermore, to mitigate redundant learning in the backbone branch of the residual structure, we introduce the concept of XOR meta-residuals, i.e., selecting pre-learning residuals using the Exclusive-OR (XOR) operation for the backbone branch. Finally, by integrating the OA shortcut and XOR meta-residuals, we devise the XOR residual block and further construct XOResNet with varying depths based on this block. Extensive experiments on four datasets, Fashion-MNIST, CIFAR-10, CIFAR-100, and miniImageNet, show that the proposed XOResNet outperforms existing state-of-the-art deep SNNs optimized via gradient descent. These results validate the effectiveness of our OA shortcut and XOR meta-residual components in overcoming fundamental limitations of residual learning in SNNs, providing new architectural insights for building high-performance neuromorphic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。