arXiv:2606.14094cs.CVcs.AI2026-06

融合帧图与事件相机数据,提升复杂场景下多目标跟踪性能

FEMOT: Multi-Object Tracking using Frame and Event Cameras

论文配图:FEMOT: Multi-Object Tracking using Frame and Event Cameras
图 1 · 摘自论文原文
  • 分离处理RGB与事件数据,在频域融合以互补优势
  • 在FEMOT和DSEC-MOT上实现更稳定的定位与身份关联
  • 构建首个大规模双模态跟踪数据集,支持系统性评估

传统RGB相机虽能捕捉丰富外观与语义信息,但在运动模糊、低光照、过曝等复杂场景下性能下降。生物启发的事件相机具备高时间分辨率和高动态范围,可在极端条件下提供补充信息。然而,由于缺乏大规模且标注良好的数据集,RGB-事件多目标跟踪仍处于探索阶段。为此,我们提出FEMOT——一个涵盖多样真实场景与14个挑战属性的大规模RGB-事件多目标跟踪数据集。该数据集同时包含RGB与事件数据及高质量标注,为系统评估多模态跟踪方法提供了可靠平台。基于FEMOT,我们重新训练并评估了十余种强追踪器,建立了全面基准。此外,我们提出FEMOTR框架,通过解耦RGB与事件特征,并在频域融合,有效利用二者互补特性,实现鲁棒的目标定位与身份关联。在FEMOT与DSEC-MOT上的大量实验验证了方法的有效性。源代码与数据集已开源于https://github.com/Event-AHU/FEMOT。

原文摘要 · Abstract (English)

Conventional RGB cameras have been widely used in multi-object tracking due to their ability to capture rich appearance and semantic information. However, their performance is often degraded under complex real-world challenges, such as motion blur, low illumination, and overexposure. Bio-inspired event cameras offer high temporal resolution and high dynamic range, providing complementary cues under extreme scenarios. Nevertheless, RGB-event multi-object tracking remains underexplored due to the lack of large-scale and well-annotated datasets. To address this issue, we propose FEMOT, a large-scale RGB-event multi-object tracking dataset that covers diverse real-world scenarios and 14 challenging attributes. With both RGB and event data as well as high-quality annotations, FEMOT provides a reliable platform for systematically evaluating RGB-event multi-object tracking methods. Based on FEMOT, we retrain and evaluate over ten strong trackers, thereby establishing a comprehensive benchmark for future research. Furthermore, we propose FEMOTR, a multimodal tracking framework that decouples RGB and event features and fuses them in the frequency domain, thereby effectively exploiting their complementary characteristics for robust object localization and identity association. Extensive experiments on FEMOT and DSEC-MOT datasets demonstrate the effectiveness of the proposed method. The source code and benchmark dataset have been released on https://github.com/Event-AHU/FEMOT.

多目标跟踪事件相机双模态融合数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。