研究CLIP在军事车辆识别中的遮挡鲁棒性，发现微调可显著提升抗遮挡能力。
Occlusion Robustness of CLIP for Military Vehicle Classification
- 对比Transformer与CNN结构，前者在遮挡下表现更优
- 分散小遮挡比大块连续遮挡更影响识别准确率
- 微调模型后，遮挡超过60%才出现性能骤降，适合军事部署
视觉语言模型（如CLIP）通过图像与文本在共享嵌入空间中的对齐实现零样本分类，在标注数据稀缺的国防场景中具有优势。然而，其在部分遮挡和信噪比低等复杂军事环境下的鲁棒性尚未充分研究。本文使用包含18类军事车辆的自定义数据集，通过归一化曲线下面积（NAUC）评估不同遮挡比例下的性能。主要发现：(1) 基于Transformer的CLIP模型始终优于基于CNN的版本；(2) 细粒度、分散式遮挡比大范围连续遮挡更严重损害性能；(3) 尽管线性探针模型准确率较高，但在约35%遮挡时性能急剧下降；(4) 通过微调模型主干网络，该性能下降阈值提升至60%以上。结果表明，训练阶段需引入特定遮挡增强，并进一步探索像素级敏感性与架构韧性以支持实际部署。
原文摘要 · Abstract (English)
Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space, offering advantages for defense applications with scarce labeled data. However, CLIP's robustness in challenging military environments, with partial occlusion and degraded signal-to-noise ratio (SNR), remains underexplored. We investigate CLIP variants' robustness to occlusion using a custom dataset of 18 military vehicle classes and evaluate using Normalized Area Under the Curve (NAUC) across occlusion percentages. Four key insights emerge: (1) Transformer-based CLIP models consistently outperform CNNs, (2) fine-grained, dispersed occlusions degrade performance more than larger contiguous occlusions, (3) despite improved accuracy, performance of linear-probed models sharply drops at around 35% occlusion, (4) by finetuning the model's backbone, this performance drop occurs at more than 60% occlusion. These results underscore the importance of occlusion-specific augmentations during training and the need for further exploration into patch-level sensitivity and architectural resilience for real-world deployment of CLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。