arXiv:2410.02249cs.CVcs.NE2024-10NeurIPS被引 11

用脉冲神经网络动态切分事件流,提升视觉任务精度

Spiking Neural Network as Adaptive Event Stream Slicer

  • 用低功耗脉冲神经网络动态触发事件切分
  • 在目标追踪与识别任务中显著提升性能
  • 适合需要低功耗实时处理的智能视觉系统

事件相机因其丰富的边缘信息、高动态范围和高时间分辨率而受到广泛关注。现有许多先进事件处理算法依赖于将事件固定分组,导致在不同运动场景(如高速/低速)下丢失关键时间信息。本文提出SpikeSlicer,一种可即插即用的自适应事件流切分方法。该方法利用低功耗脉冲神经网络(SNN)触发事件切分,并设计了脉冲位置感知损失(SPA-Loss)以引导神经元在最优时间步放电。此外,提出反馈更新训练策略,通过下游人工神经网络(ANN)的反馈优化切分决策。大量实验表明,该方法在事件驱动的目标追踪与识别任务中取得显著性能提升。SpikeSlicer开创了一种新型的SNN-ANN协同范式:SNN作为高效低功耗数据处理器,协助ANN提升下游任务表现,为智能视觉系统提供了新思路与探索方向。代码已开源:https://github.com/AndyCao1125/SpikeSlicer。

原文摘要 · Abstract (English)

Event-based cameras are attracting significant interest as they provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (\eg, high/low speed).In this work, we propose SpikeSlicer, a novel-designed plug-and-play event processing method capable of splitting events stream adaptively.SpikeSlicer utilizes a low-energy spiking neural network (SNN) to trigger event slicing. To guide the SNN to fire spikes at optimal time steps, we propose the Spiking Position-aware Loss (SPA-Loss) to modulate the neuron's state. Additionally, we develop a Feedback-Update training strategy that refines the slicing decisions using feedback from the downstream artificial neural network (ANN). Extensive experiments demonstrate that our method yields significant performance improvements in event-based object tracking and recognition. Notably, SpikeSlicer provides a brand-new SNN-ANN cooperation paradigm, where the SNN acts as an efficient, low-energy data processor to assist the ANN in improving downstream performance, injecting new perspectives and potential avenues of exploration. Our code is available at https://github.com/AndyCao1125/SpikeSlicer.

脉冲神经网络事件相机自适应切分低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。