通过稀疏融合提升红外可见光目标检测效率,省时省力还准。
Efficient RGB-T Object Detection via Sparse Cross-Modality Fusion

- 先快速扫描找候选框,再对少数关键区域做跨模态融合
- 参数量少、计算成本低,高分辨率图像仍能保持高效
- 适合需要实时处理的多模态视觉系统
RGB-T检测器利用可见光与热红外模态的互补优势,在复杂环境下表现稳健。但多数方法采用双主干网络并在全图进行密集跨模态融合,导致计算开销过高。我们观察到图像中大多数区域为平滑背景(如天空、地面),可由轻量单模态模型处理。基于此,提出一种稀疏融合机制:先快速扫描图像生成候选区域,再对少量剩余候选框进行特征融合精检。设计两阶段框架实现该机制:第一阶段为轻量且模态特异的检测,生成高召回率的RoI;第二阶段通过融合驱动的检查与优化,剔除误报并精修边界框。该设计使模型自适应将更多算力分配给潜在前景,兼顾效率与精度。大量实验表明,本方法在参数更少、成本更低的前提下达到竞争性性能,并具备强高分辨率扩展能力。
原文摘要 · Abstract (English)
RGB-T detectors leverage the complementary strengths of visible and thermal infrared modalities, achieving robust performance under challenging conditions. Many of them resort to heavy dual backbones and exhaustive cross-modality fusion across the entire image, leading to impractically high computational costs. We observe that most image regions are smooth backgrounds (e.g., sky, ground) that can be easily handled by lightweight single-modality models. In light of this observation, we propose a sparse fusion mechanism for efficient RGB-T detection: first rapidly scanning the image to identify the proposals and then carefully examining the remaining sparse proposals via feature fusion. We propose a two-stage framework to instantiate this mechanism, which performs detection in two stages: 1) a lightweight and modality-specific detection stage that produces high-recall RoIs, and 2) a fusion-driven examination and refinement stage that filters out the false positives and refines the bounding boxes. This design enables the detector to adaptively allocate more computational resources to the potential foregrounds, improving the efficiency while ensuring detection accuracy. Extensive experiments show that our method achieves competitive performance with substantially fewer parameters and lower cost, while maintaining strong scalability to high-resolution images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。