arXiv:2410.20953cs.CV2024-10被引 5

构建多传感器无人机感知数据集,提升低光与复杂环境下的检测鲁棒性。

IndraEye: Infrared Electro-Optical UAV-based Perception Dataset for Robust Downstream Tasks

  • 融合红外与可见光图像,覆盖多种飞行高度与视角。
  • 含5612张图像、14.5万实例,覆盖印度次大陆多时相多背景场景。
  • 适合研究多模态学习、域适应及复杂环境下航空目标检测者。

深度神经网络在光照充足的可见光图像上表现优异,但关键应用如空中感知中,需在低光等极端条件下保持稳定性能。现有方法多关注光照或风格变化带来的域偏移,而空中感知还面临尺度随高度和俯仰角变化的挑战,以及相关性偏移问题。本文提出IndraEye数据集,为多模态(可见光-红外)无人机感知任务设计,包含5,612张图像、145,666个实例,涵盖多种视角、飞行高度、七类背景及不同时间段,覆盖印度次大陆区域。该数据集支持多模态学习、域适应、目标检测与语义分割等任务。通过基准测试验证其有效性,源代码与数据已公开于https://bit.ly/indraeye。

原文摘要 · Abstract (English)

Deep neural networks (DNNs) have shown exceptional performance when trained on well-illuminated images captured by Electro-Optical (EO) cameras, which provide rich texture details. However, in critical applications like aerial perception, it is essential for DNNs to maintain consistent reliability across all conditions, including low-light scenarios where EO cameras often struggle to capture sufficient detail. Additionally, UAV-based aerial object detection faces significant challenges due to scale variability from varying altitudes and slant angles, adding another layer of complexity. Existing methods typically address only illumination changes or style variations as domain shifts, but in aerial perception, correlation shifts also impact DNN performance. In this paper, we introduce the IndraEye dataset, a multi-sensor (EO-IR) dataset designed for various tasks. It includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent. The dataset opens up several research opportunities, such as multimodal learning, domain adaptation for object detection and segmentation, and exploration of sensor-specific strengths and weaknesses. IndraEye aims to advance the field by supporting the development of more robust and accurate aerial perception systems, particularly in challenging conditions. IndraEye dataset is benchmarked with object detection and semantic segmentation tasks. Dataset and source codes are available at https://bit.ly/indraeye.

无人机感知多模态数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。