解决红外与可见光图像检测中的极端不平衡问题
Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance
- 设计双分支检测架构，动态调整模态权重以适应质量差异
- 通过伪退化模拟真实场景，使模型在严重失真下仍保持55%的漏检率下降
- 适合应对传感器故障或恶劣环境下的多模态目标检测任务
RGB-Thermal（RGB-T）目标检测利用热红外（TIR）图像补充可见光数据，提升复杂环境下的鲁棒性。传统RGB-T检测器假设训练数据中两种模态平衡，但在真实场景中，环境或技术因素导致模态退化，引发测试时分布外（OOD）问题及训练时收敛困难。本文提出一种基-辅助检测器架构，引入模态交互模块，根据模态质量自适应加权，并有效处理不平衡样本。同时，采用模态伪退化策略，在训练中模拟真实世界中的不平衡情况。基检测器在高质量样本对上训练，为辅助检测器提供一致性约束，后者接收退化样本。该框架显著提升模型鲁棒性，确保在严重模态退化下仍保持可靠性能。实验表明，本方法能有效应对极端模态不平衡（漏检率降低55%），并提升多种基线检测器的表现。
原文摘要 · Abstract (English)
RGB-Thermal (RGB-T) object detection utilizes thermal infrared (TIR) images to complement RGB data, improving robustness in challenging conditions. Traditional RGB-T detectors assume balanced training data, where both modalities contribute equally. However, in real-world scenarios, modality degradation-due to environmental factors or technical issues-can lead to extreme modality imbalance, causing out-of-distribution (OOD) issues during testing and disrupting model convergence during training. This paper addresses these challenges by proposing a novel base-and-auxiliary detector architecture. We introduce a modality interaction module to adaptively weigh modalities based on their quality and handle imbalanced samples effectively. Additionally, we leverage modality pseudo-degradation to simulate real-world imbalances in training data. The base detector, trained on high-quality pairs, provides a consistency constraint for the auxiliary detector, which receives degraded samples. This framework enhances model robustness, ensuring reliable performance even under severe modality degradation. Experimental results demonstrate the effectiveness of our method in handling extreme modality imbalances~(decreasing the Missing Rate by 55%) and improving performance across various baseline detectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。