解决可见光与热成像中显著目标不一致问题,提升多模态检测精度。
RSONet: Region-guided Selective Optimization Network for RGB-T Salient Object Detection
- 通过区域引导机制生成相似性图,指导跨模态特征融合。
- 在多个数据集上超越27种先进方法,最高达mAP 89.3%。
- 适合做红外与可见光融合的显著目标检测任务研究者参考。
本文针对可见光(RGB)与热成像(Thermal)图像中显著区域不一致的问题,提出区域引导的选择性优化网络(RSONet),包含区域引导阶段和显著性生成阶段。在区域引导阶段,设计三个具有相同编码器-解码器结构的并行分支,配备上下文交互(CI)模块与空间感知融合(SF)模块,生成引导图以计算相似性得分。随后,在显著性生成阶段,基于先前获得的相似性值,使用选择性优化(SO)模块融合RGB与热成像特征,缓解两模态间显著目标分布不一致的影响。为进一步提升检测质量,采用密集细节增强(DDE)模块,通过多重密集连接与视觉状态空间块优化低层特征细节;同时,在高层特征中引入互作用语义(MIS)模块,通过双向融合策略挖掘位置线索。在多个公开的RGB-T数据集上进行了大量实验,结果表明,所提RSONet在性能上优于27种前沿显著目标检测方法。
原文摘要 · Abstract (English)
This paper focuses on the inconsistency in salient regions between RGB and thermal images. To address this issue, we propose the Region-guided Selective Optimization Network for RGB-T Salient Object Detection, which consists of the region guidance stage and saliency generation stage. In the region guidance stage, three parallel branches with same encoder-decoder structure equipped with the context interaction (CI) module and spatial-aware fusion (SF) module are designed to generate the guidance maps which are leveraged to calculate similarity scores. Then, in the saliency generation stage, the selective optimization (SO) module fuses RGB and thermal features based on the previously obtained similarity values to mitigate the impact of inconsistent distribution of salient targets between the two modalities. After that, to generate high-quality detection result, the dense detail enhancement (DDE) module which adopts the multiple dense connections and visual state space blocks is applied to low-level features for optimizing the detail information. In addition, the mutual interaction semantic (MIS) module is placed in the high-level features to dig the location cues by the mutual fusion strategy. We conduct extensive experiments on the RGB-T dataset, and the results demonstrate that the proposed RSONet achieves competitive performance against 27 state-of-the-art SOD methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。