用结构光让事件相机实现彩色深度感知，实时性超1400帧每秒。
E-RGB-D: Real-Time Event-Based Perception with Structured Light
- 用DLP投影生成结构光，与事件相机融合实现彩色与深度分离感知
- 色度检测达1400帧/秒，像素级深度检测达4千赫兹
- 适合机器人、三维重建等对实时性和精度要求高的场景
事件相机（ECs）作为类生物传感器，异步报告像素亮度变化，具备极高的速度和效率。尽管具有高动态范围、高时间分辨率、低功耗和计算简单等优点，传统单色事件相机在检测静态或慢速运动物体时存在局限，且缺乏颜色信息。为此，我们提出一种新方法，集成数字光处理（DLP）投影仪，构建主动结构光（ASL）实现RGB-D感知。结合事件相机与投影技术的优势，该方法可分别获取每个像素的颜色与深度信息。动态投影调节优化带宽，实现选择性彩色数据采集，生成无空间分辨率损失的彩色点云。通过商用TI LightCrafter 4500投影仪与单目单色事件相机的配合，不仅支持无帧率限制的RGB-D感知，还达成显著性能突破：颜色检测速度达1400 fps，像素级深度检测达4 kHz，大幅推进计算机视觉在机器人、3D重建等领域的应用。代码已开源：https://github.com/MISTLab/event_based_rgbd_ros
原文摘要 · Abstract (English)
Event-based cameras (ECs) have emerged as bio-inspired sensors that report pixel brightness changes asynchronously, offering unmatched speed and efficiency in vision sensing. Despite their high dynamic range, temporal resolution, low power consumption, and computational simplicity, traditional monochrome ECs face limitations in detecting static or slowly moving objects and lack color information essential for certain applications. To address these challenges, we present a novel approach that integrates a Digital Light Processing (DLP) projector, forming Active Structured Light (ASL) for RGB-D sensing. By combining the benefits of ECs and projection-based techniques, our method enables the detection of color and the depth of each pixel separately. Dynamic projection adjustments optimize bandwidth, ensuring selective color data acquisition and yielding colorful point clouds without sacrificing spatial resolution. This integration, facilitated by a commercial TI LightCrafter 4500 projector and a monocular monochrome EC, not only enables frameless RGB-D sensing applications but also achieves remarkable performance milestones. With our approach, we achieved a color detection speed equivalent to 1400 fps and 4 kHz of pixel depth detection, significantly advancing the realm of computer vision across diverse fields from robotics to 3D reconstruction methods. Our code is publicly available: https://github.com/MISTLab/event_based_rgbd_ros
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。