融合事件相机与RGB图像,提升无人机检测在复杂光照下的准确性
Neuromorphic Drone Detection: an Event-RGB Multimodal Approach
- 用事件流与RGB图像双模态数据联合建模
- 在低光照和高速运动场景下检测精度显著优于单一模态
- 适合需要鲁棒视觉感知的安防与空管系统
近年来,无人机检测成为关注热点:小型快速移动的物体可能被用于恶意或恐怖活动,亟需精确且可靠的检测系统。尽管基于RGB图像的目标检测研究丰富,但在无人机检测中仍存在局限,尤其在高动态范围、弱光等挑战性场景下。事件相机能有效保留高速运动和低光照条件下的时空信息,但对静止目标敏感度下降。为此,本文提出一种新颖的多模态融合模型,结合事件流与RGB图像的优势,实现互补增强。同时,我们发布了NeRDD(Neuromorphic-RGB Drone Detection)数据集,包含超过3.5小时的时空同步事件-RGB标注数据,支持多模态研究。
原文摘要 · Abstract (English)
In recent years, drone detection has quickly become a subject of extreme interest: the potential for fast-moving objects of contained dimensions to be used for malicious intents or even terrorist attacks has posed attention to the necessity for precise and resilient systems for detecting and identifying such elements. While extensive literature and works exist on object detection based on RGB data, it is also critical to recognize the limits of such modality when applied to UAVs detection. Detecting drones indeed poses several challenges such as fast-moving objects and scenes with a high dynamic range or, even worse, scarce illumination levels. Neuromorphic cameras, on the other hand, can retain precise and rich spatio-temporal information in situations that are challenging for RGB cameras. They are resilient to both high-speed moving objects and scarce illumination settings, while prone to suffer a rapid loss of information when the objects in the scene are static. In this context, we present a novel model for integrating both domains together, leveraging multimodal data to take advantage of the best of both worlds. To this end, we also release NeRDD (Neuromorphic-RGB Drone Detection), a novel spatio-temporally synchronized Event-RGB Drone detection dataset of more than 3.5 hours of multimodal annotated recordings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。