用事件相机和脉冲神经网络实现低功耗机器人跑酷
ES-Parkour: Advanced Robot Parkour with Bio-inspired Event Camera and Spiking Neural Network
- 用事件相机捕捉动态视觉，脉冲神经网络高效处理脉冲信号
- 跑酷任务能耗仅为传统模型的11.7%，降低88.3%
- 适合低功耗、高动态环境下的机器人智能控制应用
近年来，四足机器人在感知与运动控制方面取得显著进展，尤其借助强化学习实现了复杂环境中的复杂动作。视觉传感器如深度相机虽提升稳定性和鲁棒性，但存在工作频率低于关节控制频率、对光照敏感等局限，限制了其户外部署。此外，传感器与控制系统中的深度神经网络也带来高计算开销。为解决这些问题，本文引入脉冲神经网络（SNNs）与事件相机，完成一项具有挑战性的四足机器人跑酷任务。事件相机捕捉动态视觉数据，而SNNs能高效处理脉冲序列，模拟生物感知机制。实验表明，该方法显著优于传统模型，在跑酷任务中仅需人工神经网络（ANN）模型11.7%的能耗，实现88.3%的能效提升。通过融合事件相机与SNN，本工作推动了机器人强化学习的发展，并为复杂环境应用开辟新路径。
原文摘要 · Abstract (English)
In recent years, quadruped robotics has advanced significantly, particularly in perception and motion control via reinforcement learning, enabling complex motions in challenging environments. Visual sensors like depth cameras enhance stability and robustness but face limitations, such as low operating frequencies relative to joint control and sensitivity to lighting, which hinder outdoor deployment. Additionally, deep neural networks in sensor and control systems increase computational demands. To address these issues, we introduce spiking neural networks (SNNs) and event cameras to perform a challenging quadruped parkour task. Event cameras capture dynamic visual data, while SNNs efficiently process spike sequences, mimicking biological perception. Experimental results demonstrate that this approach significantly outperforms traditional models, achieving excellent parkour performance with just 11.7% of the energy consumption of an artificial neural network (ANN)-based model, yielding an 88.3% energy reduction. By integrating event cameras with SNNs, our work advances robotic reinforcement learning and opens new possibilities for applications in demanding environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。