在芯片上同时运行多个神经形态组件，实现低功耗高实时的机器人视觉控制。
The More the Merrier: Running Multiple Neuromorphic Components On-Chip for Robotic Control
- 用脉冲神经状态机协调多个复杂网络在芯片上并行运行。
- 在英特尔Loihi 2芯片上实现毫瓦级功耗、接近当前最优的延迟性能。
- 首次完整实现在真实机械臂上完成插拔任务，适合智能机器人系统研发者。
长期以来，神经形态硬件在机器人领域展现出低功耗、低延迟及独特学习方法的优势。然而，在处理多模态数据的复杂任务时，由于难以在神经形态硬件上协调多个网络而不得不依赖外部逻辑管理，成为主要障碍。为此，本文首次展示了一个基于视觉的机器人控制流水线，通过脉冲神经状态机实现多个复杂网络在芯片上的全硬件协同运行。该方案在英特尔Loihi 2研究芯片上验证，所有组件可在毫瓦级功耗下并发运行，延迟达到当前最先进水平。仿真中等效网络成功完成机械臂插拔任务；流水线核心模块也在真实机械臂上测试成功。
原文摘要 · Abstract (English)
It has long been realized that neuromorphic hardware offers benefits for the domain of robotics such as low energy, low latency, as well as unique methods of learning. In aiming for more complex tasks, especially those incorporating multimodal data, one hurdle continuing to prevent their realization is an inability to orchestrate multiple networks on neuromorphic hardware without resorting to off-chip process management logic. To address this, we show a first example of a pipeline for vision-based robot control in which numerous complex networks can be run entirely on hardware via the use of a spiking neural state machine for process orchestration. The pipeline is validated on the Intel Loihi 2 research chip. We show that all components can run concurrently on-chip in the milli Watt regime at latencies competitive with the state-of-the-art. An equivalent network on simulated hardware is shown to accomplish robotic arm plug insertion in simulation, and the core elements of the pipeline are additionally tested on a real robotic arm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。