arXiv:2412.02890cs.CV2024-12ICCV被引 12

用现有图像检测模型适配事件相机,性能超专用方法。

EvRT-DETR: Latent Space Adaptation of Image Detectors for Event-based Vision

  • 将图像检测器的潜空间改造,实现对事件数据的高效处理
  • 在Gen1和1Mpx/Gen4数据集上分别提升2.3和1.4 mAP
  • 无需复杂架构,适合快速部署到其他时序视觉任务

事件相机(EBC)作为类生物相机,具备低功耗、高时间分辨率和宽动态范围的优势。但其稀疏异步的数据特性使分析困难。本文提出I2EvDet框架,将主流图像检测器迁移至事件数据。首先证明,仅用图像化表示的EBC数据训练实时检测变压器(RT-DETR),即可达到与专用方法相当的性能。其次,通过微小结构调整,对冻结的潜空间进行适配,实现从图像检测器到事件检测模型的转换。所提EvRT-DETR在标准基准数据集Gen1(mAP +2.3)和1Mpx/Gen4(mAP +1.4)上达到新纪录。该方法为事件视觉检测提供了一种基于主流架构的高效适配路径,可推广至其他时序视觉场景。代码已开源。

原文摘要 · Abstract (English)

Event-based cameras (EBCs) have emerged as a bio-inspired alternative to traditional cameras, offering advantages in power efficiency, temporal resolution, and high dynamic range. However, the development of image analysis methods for EBCs is challenging due to the sparse and asynchronous nature of the data. This work addresses the problem of object detection for EBC cameras. The current approaches to EBC object detection focus on constructing complex data representations and rely on specialized architectures. We introduce I2EvDet (Image-to-Event Detection), a novel adaptation framework that bridges mainstream object detection with temporal event data processing. First, we demonstrate that a Real-Time DEtection TRansformer, or RT-DETR, a state-of-the-art natural image detector, trained on a simple image-like representation of the EBC data achieves performance comparable to specialized EBC methods. Next, as part of our framework, we develop an efficient adaptation technique that transforms image-based detectors into event-based detection models by modifying their frozen latent representation space through minimal architectural additions. The resulting EvRT-DETR model reaches state-of-the-art performance on the standard benchmark datasets Gen1 (mAP $+2.3$) and 1Mpx/Gen4 (mAP $+1.4$). These results demonstrate a fundamentally new approach to EBC object detection through principled adaptation of mainstream architectures, offering an efficient alternative with potential applications to other temporal visual domains. The code is available at: https://github.com/realtime-intelligence/evrt-detr

事件视觉目标检测潜空间适配Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。