arXiv:2412.12525cs.CVcs.AI2024-12中稿 · AAAI被引 8

提出高效脉冲驱动框架CREST,提升事件相机目标检测速度与能效。

CREST: An Efficient Conjointly-trained Spike-driven Framework for Event-based Object Detection Exploiting Spatiotemporal Dynamics

  • 采用联合学习规则加速脉冲网络训练,缓解梯度消失问题。
  • 在三个数据集上实现比顶尖SNN算法高100倍的能效优势。
  • 适合部署于脉冲神经网络硬件,兼顾精度与实时性。

事件相机具有高时间分辨率、宽动态范围和低功耗特性,适用于高速与低光照下的目标检测。脉冲神经网络(SNN)因其脉冲特性在事件感知任务中前景广阔,但现有方法缺乏高效训练方式,导致梯度消失与计算复杂度高,尤其在深层网络中更为严重。同时,现有SNN框架难以有效处理多尺度时空特征,引发数据冗余并降低准确率。为此,本文提出CREST——一种新型联合训练的脉冲驱动框架,充分利用事件数据中的时空动态信息。通过引入联合学习规则,加速SNN学习并缓解梯度消失;支持双运行模式,适配不同硬件平台。框架包含全脉冲驱动的多尺度时空事件积分器(MESTOR)与时空IoU(ST-IoU)损失函数。实验表明,该方法在三个数据集上达到优异的目标识别与检测性能,相比现有最优SNN算法最高提升100倍能效,为SNN硬件实现提供高效解决方案。

原文摘要 · Abstract (English)

Event-based cameras feature high temporal resolution, wide dynamic range, and low power consumption, which is ideal for high-speed and low-light object detection. Spiking neural networks (SNNs) are promising for event-based object recognition and detection due to their spiking nature but lack efficient training methods, leading to gradient vanishing and high computational complexity, especially in deep SNNs. Additionally, existing SNN frameworks often fail to effectively handle multi-scale spatiotemporal features, leading to increased data redundancy and reduced accuracy. To address these issues, we propose CREST, a novel conjointly-trained spike-driven framework to exploit spatiotemporal dynamics in event-based object detection. We introduce the conjoint learning rule to accelerate SNN learning and alleviate gradient vanishing. It also supports dual operation modes for efficient and flexible implementation on different hardware types. Additionally, CREST features a fully spike-driven framework with a multi-scale spatiotemporal event integrator (MESTOR) and a spatiotemporal-IoU (ST-IoU) loss. Our approach achieves superior object recognition & detection performance and up to 100X energy efficiency compared with state-of-the-art SNN algorithms on three datasets, providing an efficient solution for event-based object detection algorithms suitable for SNN hardware implementation.

事件相机脉冲神经网络目标检测能效优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。