用通用分割模型提升红外小目标检测,轻量高效且性能超越现有方法。
Unleashing the Power of Generic Segmentation Models: A Simple Baseline for Infrared Small Target Detection
- 将通用分割模型适配至红外小目标检测,采用轻量蒸馏策略提升性能。
- 在NUDT和IRSTD1k数据集上分别超过SAM 14和4 IoU,准确率与吞吐量双优。
- 适合红外图像处理、小目标检测研究者快速部署与对比实验。
深度学习的进展显著推动了红外小目标检测(IRSTD)领域的发展。尽管成果显著,但当前的IRSTD方法与自然图像领域的通用分割方法之间仍存在明显差距,主要源于模态差异及红外数据稀缺。本研究通过探索通用分割模型(如Segment Anything Model, SAM)在IRSTD任务中的适应性,揭示了这些模型具备媲美顶尖方法的潜力,但尚未被充分挖掘。为此,我们提出一种简单、轻量且高效的基准模型,用于红外小目标分割。通过合理的知识蒸馏策略,使小型学生模型超越主流方法,甚至优于微调后的教师模型。此外,引入包含密集与稀疏查询的新设计,有效编码多尺度特征。在四个主流红外小目标检测数据集上的大量实验表明,该模型在精度和推理速度方面均显著优于现有方法,在NUDT上超越SAM和Semantic-SAM超过14 IoU,IRSTD1k上超越4 IoU。源代码与模型将开源。
原文摘要 · Abstract (English)
Recent advancements in deep learning have greatly advanced the field of infrared small object detection (IRSTD). Despite their remarkable success, a notable gap persists between these IRSTD methods and generic segmentation approaches in natural image domains. This gap primarily arises from the significant modality differences and the limited availability of infrared data. In this study, we aim to bridge this divergence by investigating the adaptation of generic segmentation models, such as the Segment Anything Model (SAM), to IRSTD tasks. Our investigation reveals that many generic segmentation models can achieve comparable performance to state-of-the-art IRSTD methods. However, their full potential in IRSTD remains untapped. To address this, we propose a simple, lightweight, yet effective baseline model for segmenting small infrared objects. Through appropriate distillation strategies, we empower smaller student models to outperform state-of-the-art methods, even surpassing fine-tuned teacher results. Furthermore, we enhance the model's performance by introducing a novel query design comprising dense and sparse queries to effectively encode multi-scale features. Through extensive experimentation across four popular IRSTD datasets, our model demonstrates significantly improved performance in both accuracy and throughput compared to existing approaches, surpassing SAM and Semantic-SAM by over 14 IoU on NUDT and 4 IoU on IRSTD1k. The source code and models will be released at https://github.com/O937-blip/SimIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。