构建首个大规模可见光与短波红外多光谱驾驶数据集,提升恶劣天气下自动驾驶感知能力。
RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions
- 采集10万对同步对齐的RGB-SWIR图像,覆盖多种场景与天气。
- 融合双模态数据的检测模型在恶劣条件下准确率显著优于仅用可见光。
- 适合研究多光谱感知、自动驾驶鲁棒性及跨模态图像生成的团队使用。
当前自动驾驶算法严重依赖可见光谱,在雾、雨、雪、强光和高对比度等恶劣条件下性能显著下降。尽管近红外(NIR)和长波红外(LWIR)能在一定程度上增强视觉感知,但存在局限性,且缺乏大规模数据集与基准测试。短波红外(SWIR)成像相较二者具有优势,但目前尚无公开的大规模自动驾驶专用SWIR数据集。为此,我们提出RGB与SWIR多光谱驾驶数据集(RASMD),包含10万对在多样化地点、光照和天气条件下采集的同步、空间对齐的RGB-SWIR图像对。同时提供用于RGB-SWIR图像转换的子集及部分挑战性交通场景的物体检测标注,通过物体检测与跨模态图像生成实验验证了SWIR成像的有效性。实验表明,在集成框架中融合RGB与SWIR数据能显著提升检测精度,尤其在可见光传感器表现不佳的条件下。我们期望RASMD能推动自动驾驶多光谱感知与鲁棒系统的研究进展。
原文摘要 · Abstract (English)
Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions like fog, rain, snow, glare, and high contrast. Although other spectral bands like near-infrared (NIR) and long-wave infrared (LWIR) can enhance vision perception in such situations, they have limitations and lack large-scale datasets and benchmarks. Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR. However, no publicly available large-scale datasets currently incorporate SWIR data for autonomous driving. To address this gap, we introduce the RGB and SWIR Multispectral Driving (RASMD) dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs collected across diverse locations, lighting, and weather conditions. In addition, we provide a subset for RGB-SWIR translation and object detection annotations for a subset of challenging traffic scenarios to demonstrate the utility of SWIR imaging through experiments on both object detection and RGB-to-SWIR image translation. Our experiments show that combining RGB and SWIR data in an ensemble framework significantly improves detection accuracy compared to RGB-only approaches, particularly in conditions where visible-spectrum sensors struggle. We anticipate that the RASMD dataset will advance research in multispectral imaging for autonomous driving and robust perception systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。