用3D卷积和时间记忆增强脉冲网络，让其在静态目标检测上追平传统神经网络。
Enhanced Temporal Processing in Spiking Neural Networks for Static Object Detection Using 3D Convolutions
- 用3D卷积直接融合时空信息，替代传统2D卷积。
- 在COCO2017和VOC数据集上达到与ANN相当的检测性能。
- 适合关注能效与脉冲神经网络实时推理的研究者。
脉冲神经网络（SNN）具备处理时空信息的能力，具有事件驱动和能效优势。尽管直接训练的SNN在分类任务中已接近或超越传统人工神经网络（ANN），但在基于帧的静态目标检测任务（如COCO2017）上仍存在显著性能差距。为缩小这一差距，本文聚焦于增强SNN对时空信息的处理能力。现有脉冲神经元模型在时间通道间的信息传递方面存在局限，部分研究甚至发现禁用时间维度反向传播也能获得良好结果。为此，本文提出将传统2D卷积替换为3D卷积，使时间信息直接融入卷积过程，并在神经元内部引入时间信息循环机制，提升其对时序信息的利用效率。实验表明，该方法使直接训练的SNN在COCO2017和VOC数据集上的表现达到与ANN相当的水平。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are a class of network models capable of processing spatiotemporal information, with event-driven characteristics and energy efficiency advantages. Recently, directly trained SNNs have shown potential to match or surpass the performance of traditional Artificial Neural Networks (ANNs) in classification tasks. However, in object detection tasks, directly trained SNNs still exhibit a significant performance gap compared to ANNs when tested on frame-based static object datasets (such as COCO2017). Therefore, bridging this performance gap and enabling directly trained SNNs to achieve performance comparable to ANNs on these static datasets has become one of the key challenges in the development of SNNs.To address this challenge, this paper focuses on enhancing the SNN's unique ability to process spatiotemporal information. Spiking neurons, as the core components of SNNs, facilitate the exchange of information between different temporal channels during the process of converting input floating-point data into binary spike signals. However, existing neuron models still have certain limitations in the communication of temporal information. Some studies have even suggested that disabling the backpropagation in the time dimension during SNN training can still yield good training results. To improve the SNN handling of temporal information, this paper proposes replacing traditional 2D convolutions with 3D convolutions, thus directly incorporating temporal information into the convolutional process. Additionally, temporal information recurrence mechanism is introduced within the neurons to further enhance the neurons' efficiency in utilizing temporal information.Experimental results show that the proposed method enables directly trained SNNs to achieve performance levels comparable to ANNs on the COCO2017 and VOC datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。