提升脉冲网络时序建模能力,让脉冲神经网络更精准检测物体。
Temporal Dynamics Enhancer for Directly Trained Spiking Object Detectors
- 设计时序增强模块,生成随时间变化的多样化输入信号。
- 在PASCAL VOC和EvDET200K数据集上分别达到57.7%和47.6%的mAP50-95。
- 提出脉冲驱动注意力机制,能耗仅为传统注意力的24%。
脉冲神经网络(SNN)凭借其类脑的时空动态与脉冲驱动计算,成为高效节能的人工神经网络替代方案。然而,现有SNN通常直接复制输入或在固定间隔聚合输入,导致神经元在不同时间步接收几乎相同的刺激,严重限制了模型在复杂任务(如目标检测)中的表达能力。本文提出时序动态增强器(TDE),包含两个模块:脉冲编码器(SE)用于生成跨时间步的多样化输入,注意力门控模块(AGM)根据时间依赖关系指导编码器生成。为降低AGM引入的高能耗乘法操作,进一步提出脉冲驱动注意力(SDA),显著降低注意力能耗。大量实验表明,TDE可无缝集成至现有SNN检测器中,持续优于当前最优方法,在静态PASCAL VOC数据集上实现57.7%的mAP50-95,于神经形态EvDET200K数据集上达47.6%。在能耗方面,SDA仅消耗传统注意力模块0.240倍的能量。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs), with their brain-inspired spatiotemporal dynamics and spike-driven computation, have emerged as promising energy-efficient alternatives to Artificial Neural Networks (ANNs). However, existing SNNs typically replicate inputs directly or aggregate them into frames at fixed intervals. Such strategies lead to neurons receiving nearly identical stimuli across time steps, severely limiting the model's expressive power, particularly in complex tasks like object detection. In this work, we propose the Temporal Dynamics Enhancer (TDE) to strengthen SNNs' capacity for temporal information modeling. TDE consists of two modules: a Spiking Encoder (SE) that generates diverse input stimuli across time steps, and an Attention Gating Module (AGM) that guides the SE generation based on inter-temporal dependencies. Moreover, to eliminate the high-energy multiplication operations introduced by the AGM, we propose a Spike-Driven Attention (SDA) to reduce attention-related energy consumption. Extensive experiments demonstrate that TDE can be seamlessly integrated into existing SNN-based detectors and consistently outperforms state-of-the-art methods, achieving mAP50-95 scores of 57.7% on the static PASCAL VOC dataset and 47.6% on the neuromorphic EvDET200K dataset. In terms of energy consumption, the SDA consumes only 0.240 times the energy of conventional attention modules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。