构建首个大规模事件相机自动驾驶感知数据集，提升复杂光照下视觉模型性能。
Toward Deep Representation Learning for Event-Enhanced Visual Autonomous Perception: the eAP Dataset
- 构建eAP数据集，融合事件相机与传统摄像头，支持深度表征学习。
- 首次用事件数据显著提升3D车辆检测在挑战性光照下的准确率。
- 实现200帧/秒的高实时性目标时间到接触估计，适合自动驾驶系统部署。
当前视觉自主感知系统依赖深度表征学习取得优异性能，但在极端光照条件下表现不佳。事件相机虽可缓解此问题，但缺乏大规模数据集支持其在自动驾驶场景中的深度学习研究。为此，我们提出eAP（event-enhanced Autonomous Perception）数据集，是目前最大的面向自动驾驶的事件相机数据集。该数据集支持多种感知任务的深度学习研究，包括3D车辆检测和目标时间到接触（TTC）估计。基于eAP，我们首次成功利用事件数据改进主流3D车辆检测网络，在挑战性光照条件下实现性能提升。同时，eAP支持对物体TTC估计的专门表征学习研究，我们提出一种几何感知框架，训练出可在200 FPS下运行的最优事件驱动TTC估计网络。数据集、代码及预训练模型将公开供后续研究使用。
原文摘要 · Abstract (English)
Recent visual autonomous perception systems achieve remarkable performances with deep representation learning. However, they fail in scenarios with challenging illumination.While event cameras can mitigate this problem, there is a lack of a large-scale dataset to develop event-enhanced deep visual perception models in autonomous driving scenes. To address the gap, we present the eAP (event-enhanced Autonomous Perception) dataset, the largest dataset with event cameras for autonomous perception. We demonstrate how eAP can facilitate the study of different autonomous perception tasks, including 3D vehicle detection and object time-to-contact (TTC) estimation, through deep representation learning. Based on eAP, we demonstrate the ffrst successful use of events to improve a popular 3D vehicle detection network in challenging illumination scenarios. eAP also enables a devoted study of the representation learning problem of object TTC estimation. We show how a geometryaware representation learning framework leads to the best eventbased object TTC estimation network that operates at 200 FPS. The dataset, code, and pre-trained models will be made publicly available for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。