提出轻量级时序反馈模块，提升脉冲神经网络图像识别性能
STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers
- 在编码层引入浅层时序反馈机制，结合时空位置嵌入与时序反馈
- 在CIFAR-10/100和ImageNet-1K上均实现性能提升，最佳提升达+3.2%
- 可作为通用编码方案，适用于静态场景下的脉冲神经网络设计
基于Transformer的脉冲神经网络（SNN）因脉冲信号的二值特性，相比浮点人工神经网络（ANN）存在显著性能差距。现有方法通过深层反馈回路传递高层语义信息以缩小差距，但通常跨越多个深层，导致特征转换成本高、参数开销大、能耗上升及推理延迟增加。为此，本文提出浅层时序反馈（STF），一个轻量级即插即用模块，部署于编码层，包含时序-空间位置嵌入（TSPE）与时序反馈（TF）。大量实验表明，STF在不同脉冲时间步设置下，持续提升多种Transformer架构SNN在静态数据集CIFAR-10、CIFAR-100和ImageNet-1K上的表现。进一步分析显示，STF增强了脉冲模式多样性，这是性能提升的关键。对抗鲁棒性与时序敏感性评估也证明，STF优于直接编码及其变体，展现出作为静态场景新脉冲编码方案的巨大潜力。代码将在录用后公开。
原文摘要 · Abstract (English)
Transformer-based Spiking Neural Networks (SNNs) suffer from a great performance gap compared to floating-point \mbox{Artificial} Neural Networks (ANNs) due to the binary nature of spike trains. Recent efforts have introduced deep-level feedback loops to transmit high-level semantic information to narrow this gap. However, these designs often span \mbox{multiple} deep layers, resulting in costly feature transformations, higher parameter overhead, increased energy consumption, and longer inference latency. To address this issue, we propose Shallow-level Temporal Feedback (STF), a lightweight plug-and-play module for the encoding layer, which consists of Temporal-Spatial Position Embedding (TSPE) and Temporal Feedback (TF). Extensive experiments show that STF consistently improves performance across various Transformer-based SNN backbones on static datasets, including CIFAR-10, CIFAR-100, and ImageNet-1K, under different spike timestep settings. Further analysis reveals that STF enhances the diversity of spike patterns, which is key to performance gain. Moreover, evaluations on adversarial robustness and temporal sensitivity confirm that STF outperforms direct coding and its variants, highlighting its potential as a new spike encoding scheme for static scenarios. Our code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。