EsaacSim让机器人仿真支持多模态事件相机，实时生成高帧率事件流。
EsaacSim: A Multimodal Event Camera Add-on for NVIDIA Isaac Sim
- 在Isaac Sim中集成可配置事件相机，支持灰度与Bayer RGGB事件生成。
- 同步输出RGB、APS、事件、深度和IMU数据，最高达960Hz有效事件率。
- 轻量运行：单卡仅占<400MB GPU内存，适合科研与合成数据生成。
事件视觉正成为机器人感知的重要范式，但受限于传感器稀缺和缺乏集成仿真工具，其应用仍受限制。本文提出EsaacSim，一个面向NVIDIA Isaac Sim的多模态事件相机插件，支持在线模拟可配置事件相机，生成灰度与Bayer RGGB事件。该框架支持多种分辨率，通过原生ROS2接口提供同步的RGB、APS、事件、深度和IMU输出。采用运动引导帧间补全策略，在保持与Isaac Sim渲染管线兼容的前提下提升有效时间分辨率。实验表明，系统在典型机器人场景中实现多模态同步仿真，五种事件相机分辨率下有效事件率可达240至960Hz。灰度事件生成耗时6.98–27.28ms，Bayer RGGB事件为7.58–29.16ms，额外GPU内存占用低于400MB（NVIDIA RTX 4060）。结果证明EsaacSim可高效支持机器人研究中的在线多模态事件相机仿真与合成数据生成。本文发布早期版本，报告其架构与性能。
原文摘要 · Abstract (English)
Event-based vision is becoming an increasingly important sensing paradigm for robotics, yet its adoption remains limited by sensor availability and the lack of integrated simulation tools for modern robotics platforms. This paper presents EsaacSim, a multimodal event camera add-on for NVIDIA Isaac Sim that enables online simulation of configurable event cameras with grayscale and Bayer RGGB event generation. The framework supports multiple event camera resolutions and provides synchronized RGB, APS, event, depth, and IMU outputs through native ROS2 interfaces. A motion-guided frame-gap synthesis strategy further increases the effective temporal resolution while preserving compatibility with the Isaac Sim rendering pipeline. Experimental evaluation demonstrates synchronized multimodal simulation across representative robotic scenes and efficient online performance over five event camera resolutions at effective event rates from 240 to 960Hz. Event stream generation requires 6.98--27.28ms for grayscale events and 7.58--29.16ms for Bayer RGGB events while using less than 400MB of additional GPU memory on an NVIDIA RTX~4060 GPU. These results show that EsaacSim enables supports online multimodal event-camera simulation for robotics research and synthetic data generation. We release an early version of the simulator and report its current architecture and performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。