arXiv:2509.18738cs.CV2025-09被引 21

用混合提示增强分割模型,提升红外可见光图像显著目标检测精度

HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection

  • 设计动态融合网络生成高质量初始显著图作为视觉提示
  • 在三个数据集上达到当前最优性能,显著优于基线方法
  • 可插拔式设计,适配多种检测方法,适合做跨模态目标检测研究

RGB-thermal显著目标检测(RGB-T SOD)旨在融合可见光与热成像的互补信息,识别突出物体。然而,由于特征融合不足和数据稀缺等内外部限制,精确提取边界和完整目标仍具挑战。本文提出一种新型混合提示驱动的分割一切模型(HyPSAM),利用分割一切模型(SAM)的零样本泛化能力实现RGB-T SOD。首先提出动态融合网络(DFNet),通过动态卷积与多分支解码,实现自适应跨模态交互,生成高质量初始显著图作为视觉提示。此外,设计即插即用的优化网络(P2RNet),结合文本、掩码和框提示,引导SAM精炼显著图。实验在三个公开数据集上验证,本方法性能达最新水平,且具备强通用性,可无缝集成至不同RGB-T SOD方法中,显著提升效果,凸显提示工程在该领域的潜力。代码与结果见:https://github.com/milotic233/HyPSAM。

原文摘要 · Abstract (English)

RGB-thermal salient object detection (RGB-T SOD) aims to identify prominent objects by integrating complementary information from RGB and thermal modalities. However, learning the precise boundaries and complete objects remains challenging due to the intrinsic insufficient feature fusion and the extrinsic limitations of data scarcity. In this paper, we propose a novel hybrid prompt-driven segment anything model (HyPSAM), which leverages the zero-shot generalization capabilities of the segment anything model (SAM) for RGB-T SOD. Specifically, we first propose a dynamic fusion network (DFNet) that generates high-quality initial saliency maps as visual prompts. DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction, overcoming the limitations of fixed-parameter kernels and enhancing multi-modal feature representation. Moreover, we propose a plug-and-play refinement network (P2RNet), which serves as a general optimization strategy to guide SAM in refining saliency maps by using hybrid prompts. The text prompt ensures reliable modality input, while the mask and box prompts enable precise salient object localization. Extensive experiments on three public datasets demonstrate that our method achieves state-of-the-art performance. Notably, HyPSAM has remarkable versatility, seamlessly integrating with different RGB-T SOD methods to achieve significant performance gains, thereby highlighting the potential of prompt engineering in this field. The code and results of our method are available at: https://github.com/milotic233/HyPSAM.

多模态显著目标检测提示工程分割模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。