解决无人机红外可见光检测中目标位置错位问题,提升检测精度。
Mitigating the Impact of Prominent Position Shift in Drone-based RGBT Object Detection
- 将跨模态位置偏移视为标签噪声,用均值教师机制动态校正框位置
- 在无人机数据集上使感知模态准确率提升25.52 aSim点,mAP50提高8.6
- 适合做多模态无人机目标检测的开发者与研究者参考
基于无人机的RGBT目标检测在全天候应用中至关重要。然而真实场景下的无人机视角RGBT数据存在显著的位置偏移问题,即微小目标在不同模态间位置差异大。例如,热成像中目标轻微偏移会导致其在可见光图像中偏离主体。由于标注通常基于某一模态(参考模态),另一模态(感知模态)缺乏准确监督信号,阻碍检测器学习良好表征。此外,模态间特征点不匹配会使融合特征对检测头造成混淆。本文提出将跨模态框偏移问题建模为标签噪声,并通过新型均值教师-跨模态框校正头集成(CBC)在线解决。同时设计移位窗口级联对齐(SWCA)模块,挖掘移位窗口内长程依赖关系,级联对齐感知与参考模态特征。在两个无人机RGBT目标检测数据集上的实验表明,校正结果在视觉和定量上均更优。尤其,CBC模块使感知模态真值精度提升25.52 aSim点。整体检测器在RGBTDronePerson上达到43.55 mAP_50,且在DroneVehicle数据集的偏移子集上超越现有方法8.6 mAP50。代码与数据将公开。
原文摘要 · Abstract (English)
Drone-based RGBT object detection plays a crucial role in many around-the-clock applications. However, real-world drone-viewed RGBT data suffers from the prominent position shift problem, i.e., the position of a tiny object differs greatly in different modalities. For instance, a slight deviation of a tiny object in the thermal modality will induce it to drift from the main body of itself in the RGB modality. Considering RGBT data are usually labeled on one modality (reference), this will cause the unlabeled modality (sensed) to lack accurate supervision signals and prevent the detector from learning a good representation. Moreover, the mismatch of the corresponding feature point between the modalities will make the fused features confusing for the detection head. In this paper, we propose to cast the cross-modality box shift issue as the label noise problem and address it on the fly via a novel Mean Teacher-based Cross-modality Box Correction head ensemble (CBC). In this way, the network can learn more informative representations for both modalities. Furthermore, to alleviate the feature map mismatch problem in RGBT fusion, we devise a Shifted Window-Based Cascaded Alignment (SWCA) module. SWCA mines long-range dependencies between the spatially unaligned features inside shifted windows and cascaded aligns the sensed features with the reference ones. Extensive experiments on two drone-based RGBT object detection datasets demonstrate that the correction results are both visually and quantitatively favorable, thereby improving the detection performance. In particular, our CBC module boosts the precision of the sensed modality ground truth by 25.52 aSim points. Overall, the proposed detector achieves an mAP_50 of 43.55 points on RGBTDronePerson and surpasses a state-of-the-art method by 8.6 mAP50 on a shift subset of DroneVehicle dataset. The code and data will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。