用视觉变压器提升热成像武器分割，效果远超传统方法
Beyond RGB: Leveraging Vision Transformers for Thermal Weapon Segmentation
- 采用四种视觉变压器架构，捕捉热成像中长距离依赖关系
- SegFormer-b5达94.15% mIoU，b0实现98.32 FPS高速推理
- 适用于低光、遮挡等复杂安防场景，兼顾精度与速度
热成像武器分割对监控与安防至关重要，能在低光和视觉遮挡条件下实现鲁棒检测，而基于RGB的系统在此类场景中表现不佳。尽管卷积神经网络（CNN）主导热成像分割研究，但其对长程依赖和精细结构的建模能力有限。视觉变换器（ViTs）凭借全局上下文建模能力，在RGB分割任务中已达到顶尖水平，但在热武器分割中的潜力仍待挖掘。本文针对自建热成像数据集（含9,711张真实监控视频图像，通过SAM2自动标注），评估了四种变压器架构：SegFormer、DeepLabV3+、SegNeXt和Swin Transformer在二值武器分割任务中的表现。在MMSegmentation框架下采用标准增强策略，确保训练稳健性与架构间公平对比。实验结果表明，性能显著提升：SegFormer-b5取得最高mIoU（94.15%）与像素准确率（97.04%），SegFormer-b0实现最快推理速度（98.32 FPS）且具备90.84%的mIoU；SegNeXt-mscans在85.12 FPS下达到92.24% mIoU，DeepLabV3+ R101-D8在29.86 FPS下达到92.76% mIoU。变压器架构展现出对低光及遮挡环境下武器检测的强泛化能力，具备灵活的精度-速度权衡，适用于多样化的实时安防应用。
原文摘要 · Abstract (English)
Thermal weapon segmentation is crucial for surveillance and security applications, enabling robust detection under lowlight and visually obscured conditions where RGB-based systems fail. While convolutional neural networks (CNNs) dominate thermal segmentation literature, their ability to capture long-range dependencies and fine structural details is limited. Vision Transformers (ViTs), with their global context modeling capabilities, have achieved state-of-the-art results in RGB segmentation tasks, yet their potential in thermal weapon segmentation remains underexplored. This work adapts and evaluates four transformer-based architectures SegFormer, DeepLabV3\+, SegNeXt, and Swin Transformer for binary weapon segmentation on a custom thermal dataset comprising 9,711 images collected from real world surveillance videos and automatically annotated using SAM2. We employ standard augmentation strategies within the MMSegmentation framework to ensure robust model training and fair architectural comparison. Experimental results demonstrate significant improvements in segmentation performance: SegFormer-b5 achieves the highest mIoU (94.15\%) and Pixel Accuracy (97.04\%), while SegFormer-b0 provides the fastest inference speed (98.32 FPS) with competitive mIoU (90.84\%). SegNeXt-mscans offers balanced performance with 85.12 FPS and 92.24\% mIoU, and DeepLabV3\+ R101-D8 reaches 92.76\% mIoU at 29.86 FPS. The transformer architectures demonstrate robust generalization capabilities for weapon detection in low-light and occluded thermal environments, with flexible accuracy-speed trade-offs suitable for diverse real-time security applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。