用神经形态芯片实现实时避障,能耗仅传统方法的1.6%。
Fully Asynchronous Neuromorphic Perception for Mobile Robot Dodging with Loihi Chips
- 事件流逐个异步处理,模拟生物神经信号传递。
- 在不同光照下比帧方法更稳定,嵌入式能耗低至1.64%。
- 首个在真实机器人上实现的全异步神经形态系统。
自然生物中稀疏且异步的感知与处理可实现超低延迟和节能感知。事件相机(即类脑视觉传感器)旨在模仿这一特性,但如何充分利用稀疏异步事件流仍具挑战。受标准相机算法影响,现有大多数事件算法仍采用“事件组”处理范式(如事件帧、3D体素),导致特征丢失、事件堆积和高计算负担,背离了事件相机的设计初衷。为此,我们提出一种全异步神经形态范式,集成事件相机、脉冲网络与神经形态处理器(英特尔Loihi)。该范式可逐个异步处理每个事件,忠实模拟生物大脑的脉冲驱动处理机制。我们在真实移动机器人避障任务中详细对比了该范式与传统“事件组”处理范式的性能。实验表明,该方案在不同时间窗口和光照条件下均表现出更强鲁棒性;其在嵌入式Loihi处理器上的单次推理能耗仅为NVIDIA Jetson Orin NX上事件脉冲张量方法的4.30%,以及同处理器上事件帧方法的1.64%。据我们所知,这是首个在真实移动机器人上实现全异步神经形态范式解决序列任务的工作。
原文摘要 · Abstract (English)
Sparse and asynchronous sensing and processing in natural organisms lead to ultra low-latency and energy-efficient perception. Event cameras, known as neuromorphic vision sensors, are designed to mimic these characteristics. However, fully utilizing the sparse and asynchronous event stream remains challenging. Influenced by the mature algorithms of standard cameras, most existing event-based algorithms still rely on the "group of events" processing paradigm (e.g., event frames, 3D voxels) when handling event streams. This paradigm encounters issues such as feature loss, event stacking, and high computational burden, which deviates from the intended purpose of event cameras. To address these issues, we propose a fully asynchronous neuromorphic paradigm that integrates event cameras, spiking networks, and neuromorphic processors (Intel Loihi). This paradigm can faithfully process each event asynchronously as it arrives, mimicking the spike-driven signal processing in biological brains. We compare the proposed paradigm with the existing "group of events" processing paradigm in detail on the real mobile robot dodging task. Experimental results show that our scheme exhibits better robustness than frame-based methods with different time windows and light conditions. Additionally, the energy consumption per inference of our scheme on the embedded Loihi processor is only 4.30% of that of the event spike tensor method on NVIDIA Jetson Orin NX with energy-saving mode, and 1.64% of that of the event frame method on the same neuromorphic processor. As far as we know, this is the first time that a fully asynchronous neuromorphic paradigm has been implemented for solving sequential tasks on real mobile robot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。