用事件相机实现高速低延迟3D目标检测
Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras
- 首次将事件相机引入3D目标检测，利用其高时间分辨率
- 在帧间间隔也能检测，通过历史3D信息补全
- 构建首个事件相机3D检测数据集，支持100 FPS标注
点云中的3D目标检测在自动驾驶系统中至关重要。近期，融合相机信息的多模态方法取得了显著性能提升。为实现安全高效的自动驾驶，算法需兼具高精度、高速度和低延迟。然而，现有方法受限于固定帧率传感器（如激光雷达和相机）的延迟与带宽瓶颈。为此，我们首次将异步事件相机引入3D目标检测，利用其高时间分辨率和低带宽特性，实现高速3D目标检测。本方法可在同步数据缺失的帧间间隔中，通过事件相机回溯之前的3D信息完成检测。此外，我们提出了首个基于事件相机的3D目标检测数据集DSEC-3DOD，包含100 FPS的真值3D边界框，建立了事件相机3D检测的首个基准。代码与数据集已公开于https://github.com/mickeykang16/Ev3DOD。
原文摘要 · Abstract (English)
Detecting 3D objects in point clouds plays a crucial role in autonomous driving systems. Recently, advanced multi-modal methods incorporating camera information have achieved notable performance. For a safe and effective autonomous driving system, algorithms that excel not only in accuracy but also in speed and low latency are essential. However, existing algorithms fail to meet these requirements due to the latency and bandwidth limitations of fixed frame rate sensors, e.g., LiDAR and camera. To address this limitation, we introduce asynchronous event cameras into 3D object detection for the first time. We leverage their high temporal resolution and low bandwidth to enable high-speed 3D object detection. Our method enables detection even during inter-frame intervals when synchronized data is unavailable, by retrieving previous 3D information through the event camera. Furthermore, we introduce the first event-based 3D object detection dataset, DSEC-3DOD, which includes ground-truth 3D bounding boxes at 100 FPS, establishing the first benchmark for event-based 3D detectors. The code and dataset are available at https://github.com/mickeykang16/Ev3DOD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。