让可见光模型更好适应热成像,提升无人机与地面视觉系统跨域检测能力。
SAGA: Semantic-Aware Gray color Augmentation for Visible-to-Thermal Domain Adaptation across Multi-View Drone and Ground-Based Vision Systems
- 基于语义感知的灰度增强,提取热成像相关物体特征以减少颜色偏差。
- 在多视角无人机数据集上,检测准确率提升0.4%至7.6%(mAP)。
- 适合做跨模态目标检测、无人系统感知与热成像领域适应研究者。
域自适应热成像目标检测对降低可见光(RGB)到热成像(红外,IR)转换中对配准图像对的需求和减少对大规模标注红外数据集的依赖具有重要意义。然而,红外图像固有的缺乏颜色与纹理信息,使基于可见光训练的模型产生更多误检并生成低质量伪标签。为此,本文提出语义感知灰度增强(SAGA),通过提取与红外图像相关的物体级特征,缓解颜色偏差并缩小域间差距。此外,为验证SAGA在无人机影像中的效果,我们构建了多传感器(RGB-IR)数据集IndraEye,包含5,612张图像、145,666个实例,覆盖多种角度、高度、背景及昼夜场景,支持多模态学习、目标检测与分割的域自适应研究,并有助于探索传感器优劣。实验表明,将SAGA与先进域自适应方法结合,在自动驾驶和IndraEye数据集上实现0.4%至7.6%(mAP)的稳定性能提升。代码与数据集已公开于https://github.com/airliisc/IndraEye。
原文摘要 · Abstract (English)
Domain-adaptive thermal object detection plays a key role in facilitating visible (RGB)-to-thermal (IR) adaptation by reducing the need for co-registered image pairs and minimizing reliance on large annotated IR datasets. However, inherent limitations of IR images, such as the lack of color and texture cues, pose challenges for RGB-trained models, leading to increased false positives and poor-quality pseudo-labels. To address this, we propose Semantic-Aware Gray color Augmentation (SAGA), a novel strategy for mitigating color bias and bridging the domain gap by extracting object-level features relevant to IR images. Additionally, to validate the proposed SAGA for drone imagery, we introduce the IndraEye, a multi-sensor (RGB-IR) dataset designed for diverse applications. The dataset contains 5,612 images with 145,666 instances, captured from diverse angles, altitudes, backgrounds, and times of day, offering valuable opportunities for multimodal learning, domain adaptation for object detection and segmentation, and exploration of sensor-specific strengths and weaknesses. IndraEye aims to enhance the development of more robust and accurate aerial perception systems, especially in challenging environments. Experimental results show that SAGA significantly improves RGB-to-IR adaptation for autonomous driving and IndraEye dataset, achieving consistent performance gains of +0.4% to +7.6% (mAP) when integrated with state-of-the-art domain adaptation techniques. The dataset and codes are available at https://github.com/airliisc/IndraEye.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。