快速模拟神经形态机器人感知与动态，支持实时训练与测试。
Neurosim: A Fast Simulator for Neuromorphic Robot Perception
- 用GPU实现毫秒级传感器与飞行器动态仿真，最高达2700帧/秒。
- 集成零拷贝通信库Cortex，支持多模态数据同步与低延迟传输。
- 适合研究神经形态感知、自主控制算法的实时闭环验证。
Neurosim 是一个快速、实时、高性能的传感器仿真库，可模拟动态视觉传感器、RGB相机、深度传感器和惯性传感器，并在复杂动态环境中仿真多旋翼飞行器的敏捷动力学。该系统在桌面级GPU上可达到约2700帧/秒的帧率。Neurosim 通过基于 ZeroMQ 的通信库 Cortex 实现与机器学习和机器人工作流的无缝集成，Cortex 提供高吞吐、低延迟的消息传递系统，原生支持 NumPy 数组和 PyTorch 张量。本文阐述了 Neurosim 与 Cortex 的设计哲学，并展示其在（i）使用时间同步的多模态数据进行自监督学习来训练神经形态感知与控制算法，以及（ii）在闭环中测试这些算法的实时实现中的应用。Neurosim 和 Cortex 已开源，项目地址为 https://github.com/grasp-lyrl/neurosim。
原文摘要 · Abstract (English)
Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。