提出高效事件相机目标检测模型,速度更快且精度更高。
EMF: Event Meta Formers for Event-based Real-time Traffic Object Detection
- 设计专用于事件数据的进展提取模块,结合轻量卷积结构提升效率。
- 在Prophesee Gen1上比顶尖方法高1.6 mAP,推理快14%。
- 适合实时交通感知场景,尤其擅长处理稀疏事件流数据。
事件相机具有更高的时间分辨率,且相比传统RGB相机所需存储和带宽更少。然而,由于事件驱动方法性能相对滞后,尚未在自动驾驶等关键应用中替代传统相机。现有事件检测方法多采用计算量大的基于Transformer的方案,但其资源密集型组件无法有效利用事件相机的稀疏性与高时间分辨率特性,且多从视觉领域迁移而来,缺乏对事件相机的针对性设计。本文提出一种新型事件相机目标检测骨干网络EMF,引入专为事件数据设计的事件进展提取模块,并采用基于卷积的高效元变换器(Metaformer)结构。在主流交通目标检测基准上进行评估,并开展跨数据集泛化测试。结果表明,该模型在Prophesee Gen1数据集上比当前最优方法提升1.6 mAP,同时推理时间减少14%。EMF成为该领域最快的基于深度神经网络架构,显著优于多数高效事件检测模型。此外,模型展现出更强的未见数据泛化能力,且随数据量增加表现出更好的可扩展性。
原文摘要 · Abstract (English)
Event cameras have higher temporal resolution, and require less storage and bandwidth compared to traditional RGB cameras. However, due to relatively lagging performance of event-based approaches, event cameras have not yet replace traditional cameras in performance-critical applications like autonomous driving. Recent approaches in event-based object detection try to bridge this gap by employing computationally expensive transformer-based solutions. However, due to their resource-intensive components, these solutions fail to exploit the sparsity and higher temporal resolution of event cameras efficiently. Moreover, these solutions are adopted from the vision domain, lacking specificity to the event cameras. In this work, we explore efficient and performant alternatives to recurrent vision transformer models and propose a novel event-based object detection backbone. The proposed backbone employs a novel Event Progression Extractor module, tailored specifically for event data, and uses Metaformer concept with convolution-based efficient components. We evaluate the resultant model on well-established traffic object detection benchmarks and conduct cross-dataset evaluation to test its ability to generalize. The proposed model outperforms the state-of-the-art on Prophesee Gen1 dataset by 1.6 mAP while reducing inference time by 14%. Our proposed EMF becomes the fastest DNN-based architecture in the domain by outperforming most efficient event-based object detectors. Moreover, the proposed model shows better ability to generalize to unseen data and scales better with the abundance of data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。