arXiv:2501.12482cs.CVcs.ET2025-01被引 4

用事件相机实现高速低功耗目标检测与追踪,效率提升超5倍。

TOFFE -- Temporally-binned Object Flow from Events for High-speed and Energy-Efficient Object Detection and Tracking

  • 融合脉冲神经网络与传统神经网络,高效处理稀疏事件数据。
  • 在边缘设备上能耗降低5.7至8.3倍,延迟减少4.6至5.8倍。
  • 专为高速运动设计,适合无人机等资源受限的移动机器人。

目标检测与追踪是实现机器人自主导航的关键感知任务。小型无人机等边缘机器人需在资源有限下高速执行复杂动作,对算法与硬件提出严苛要求。传统帧基相机虽提供丰富空间信息且同步感测简单,但跨帧获取细节能耗高,且时间分辨率低,难以应对高速运动。事件相机通过生物启发机制,仅记录亮度变化,具备极高时间分辨率与极低功耗,适配高速场景。但其异步稀疏输出不兼容常规深度学习方法。本文提出TOFFE,一种轻量级混合框架,用于估计事件数据中的目标运动(含位姿、方向、速度),称为对象流。TOFFE结合脉冲神经网络(SNNs)与模拟神经网络(ANNs),在高时间分辨率下高效处理事件,且易于训练。此外,我们构建了一个包含高速物体运动的新型事件合成数据集以训练TOFFE。实验表明,相比先前事件基目标检测基线,TOFFE在边缘GPU(Jetson TX2)与混合硬件(Loihi-2 + Jetson TX2)上分别实现5.7倍/8.3倍能耗降低和4.6倍/5.8倍延迟减少。

原文摘要 · Abstract (English)

Object detection and tracking is an essential perception task for enabling fully autonomous navigation in robotic systems. Edge robot systems such as small drones need to execute complex maneuvers at high-speeds with limited resources, which places strict constraints on the underlying algorithms and hardware. Traditionally, frame-based cameras are used for vision-based perception due to their rich spatial information and simplified synchronous sensing capabilities. However, obtaining detailed information across frames incurs high energy consumption and may not even be required. In addition, their low temporal resolution renders them ineffective in high-speed motion scenarios. Event-based cameras offer a biologically-inspired solution to this by capturing only changes in intensity levels at exceptionally high temporal resolution and low power consumption, making them ideal for high-speed motion scenarios. However, their asynchronous and sparse outputs are not natively suitable with conventional deep learning methods. In this work, we propose TOFFE, a lightweight hybrid framework for performing event-based object motion estimation (including pose, direction, and speed estimation), referred to as Object Flow. TOFFE integrates bio-inspired Spiking Neural Networks (SNNs) and conventional Analog Neural Networks (ANNs), to efficiently process events at high temporal resolutions while being simple to train. Additionally, we present a novel event-based synthetic dataset involving high-speed object motion to train TOFFE. Our experimental results show that TOFFE achieves 5.7x/8.3x reduction in energy consumption and 4.6x/5.8x reduction in latency on edge GPU(Jetson TX2)/hybrid hardware(Loihi-2 and Jetson TX2), compared to previous event-based object detection baselines.

事件相机低功耗高速追踪神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。