提出混合注意力机制,提升复杂环境下红外可见光行人检测的鲁棒性。
Hybrid Attention for Robust RGB-T Pedestrian Detection in Real-World Conditions
- 设计混合注意力模块,动态融合部分重叠或单模态缺失的红外与可见光图像。
- 在模拟部分重叠和传感器失效场景下,性能优于现有方法。
- 适配嵌入式设备,适合自动驾驶等实时应用部署。
多光谱行人检测近年来受到广泛关注,尤其在自动驾驶领域。为应对极端光照条件挑战,红外与可见光图像融合展现出优势。然而,现有融合方法依赖于RGB-热成像(RGB-T)图像完全重叠的假设,这在真实应用中常不成立,因传感器配置导致仅部分重叠,或出现模态信息丢失。本文提出一种新型混合注意力(Hybrid Attention, HA)模块,以缓解部分重叠与传感器故障引起的性能下降问题,即场景中至少部分区域仅由单一传感器捕获。我们设计了一种改进的RGB-T融合算法,具备对实际推理中部分重叠和传感器失效的鲁棒性。同时采用轻量级骨干网络,适应嵌入式系统资源限制。通过模拟多种部分重叠与传感器失效场景进行实验,结果表明所提方法优于当前最优方法,在真实世界挑战下表现更优。
原文摘要 · Abstract (English)
Multispectral pedestrian detection has gained significant attention in recent years, particularly in autonomous driving applications. To address the challenges posed by adversarial illumination conditions, the combination of thermal and visible images has demonstrated its advantages. However, existing fusion methods rely on the critical assumption that the RGB-Thermal (RGB-T) image pairs are fully overlapping. These assumptions often do not hold in real-world applications, where only partial overlap between images can occur due to sensors configuration. Moreover, sensor failure can cause loss of information in one modality. In this paper, we propose a novel module called the Hybrid Attention (HA) mechanism as our main contribution to mitigate performance degradation caused by partial overlap and sensor failure, i.e. when at least part of the scene is acquired by only one sensor. We propose an improved RGB-T fusion algorithm, robust against partial overlap and sensor failure encountered during inference in real-world applications. We also leverage a mobile-friendly backbone to cope with resource constraints in embedded systems. We conducted experiments by simulating various partial overlap and sensor failure scenarios to evaluate the performance of our proposed method. The results demonstrate that our approach outperforms state-of-the-art methods, showcasing its superiority in handling real-world challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。